Here's a (very) quick map of the Rethil plantation, with what I consider to be key locations up for discussion:

Dotted outlines are fields/land, full lines are buildings.
Yellow lines are places that I think should no longer belong to Rethil. The plantation dock might still be useful as a ship travel location; I'd consider consolidating it with the shack village to the north.
White lines are places I could also see losing their Rethil association.
Blue lines are what I consider the core Rethil Plantation. A dock could be added to either of the marked towers, providing the Rethil Plantation with a far more convenient dock. (The current dock is roughly the same distance to the core Rethil plantation as Almas Thirr, and is a little less than half the distance from the core plantation to Andothren).
Edit: all landowners should probably own at least one slave. The individual farmers/plantation owners might also work for Hlaalu higher-ups, for example one farmer might actually work for Orvas Dren.
Edit again: alternatively, rather than creating a lot of independent or semi-independent farmhouses, the lands split off from the Rethil plantation could be consolidated into one or two still large-ish plantations. For example Mundrethi could own all of the yellow territories (which would make him a lot richer than he currently is) while a new fellow could own the white territories. Vys-Ishra would not belong to any plantation.