[Tutorial] Heightmapping Draft

1 post / 0 new
Jani's picture
Jani
Lead DeveloperDeveloperInterior DeveloperQuest Developer
Joined:
2017-10-08 01:57
Last seen:
11 hours 19 min ago

==================================
===                8-BIT ROUTE                ===
==================================
EXPORTING A HEIGHTMAP
Exporting land data can be easily done using tes3cmd. Example for some SHOTN sections:

tes3cmd dump --type LAND --raw-with-header "Map_Dump.esp" Sky_Main.esp
tes3cmd dump --type LAND --raw "Map_Dump.esp" Sky_Ds.esp
tes3cmd dump --type LAND --raw "Map_Dump.esp" Sky_Mk.esp

Then, we're going to need an 8-bit bmp from TESAnnwyn. Scaling should be done so that the highest point (z = 32768, THU = 32768/8 = 4096) is mapped to the highest value of the used data type (255 for 8-bit, 127 for 7-bit).  Don't use offsets, as we're NOT going to do ocean floor edits.
For later re-importing, we'll go with 7 bits: s = 127/4096 ~ 1/32

tesannwyn -p 2 -b 8 -s 0.03125 Map_Dump.esp

Remember the image dimensions (5888x3392) and the bottom left coordinates (-120,-14) of the image! Example data provided for SHOTN.

EDITING THE HEIGHTMAP
There are three constraints for the heightmap to keep in mind before re-importing:
1) Ocean floor should be at 0 (#000000)
2) Minimum land should be at 2 (#020202)
3) Maximum height should be at 127 (#7F7F7F)
Then, open the map in your preferred image editor, and:
1) Set the image mode to 16-bit grayscale
2) Create a new layer on top with solid fill #020202.
3) Merge down on the heightmap with linear dodge (add)
4) Paint over any mountain peak that's brighter than 127.
5) Select all ocean pixels and replace them with transparency.
Optional: Create a pure white background layer for better visibility.
6) Make your edits to the heightmap. To raise land, paint with #020202. Avoid areas with high gradients (going from white to black within a fex pixels on land). Avoid visible edges in the heightmap. Avoid using the same color too much along the edges of the map.
7) Create a new layer on the bottom with solid fill #000000
8) (This is the point of no return, so make sure to make a backup .psd of the file!) Merge the heightmap down on it.
9) Flip the map vertically and add 2px gaussian blur.
10) Save as "tesannwyn-import.raw", Photoshop RAW format, Interleaved channel order, IBM PC byte order.

KNOWN ISSUES WITH THE 8-BIT ROUTE
Your exported heightmap is going to be 0-127 instead of 0-255. This means two things:
1) You're losing accuracy in the export (about 15-20%), so rivers might dry up, or mountains might get lower. This shouldn't remain an issue after editing the heightmap.
2) You're going to have a hard time editing the map, because our eyes aren't very good at dealing with this color range.
If you have a way to manually downscale ("darken") an 8-bit grayscale bitmap to 7 bits without messing up the overall scale of the map, share your wisdom with us on Discord.
Another known issue is that the mountains are too short (sometimes even by 20%). This needs more investigation into the scaling system on TESAnnwyn. You can use larger scaling factors if you'd like to (such as 0.1494) but that's going to introduce gradient errors to the conversion at the cost of only decreasing the maximum height errors to about 10%.

==================================
===               16-BIT ROUTE               ===
==================================
EXPORTING A HEIGHTMAP
Exporting land data can be easily done using tes3cmd. Example for some SHOTN sections:

tes3cmd dump --type LAND --raw-with-header "Map_Dump.esp" Sky_Main.esp
tes3cmd dump --type LAND --raw "Map_Dump.esp" Sky_Ds.esp
tes3cmd dump --type LAND --raw "Map_Dump.esp" Sky_Mk.esp

Then, we're going to need a 16-bit raw file from TESAnnwyn. Scaling should be done so that the highest point (z = 32768, THU = 32768/8 = 4096) is mapped to the highest value of the useable Photoshop data type (32768 for 15 bits).  Don't use offsets, as we're NOT going to do ocean floor edits. The factor is: s = 32768/4096 = 8

tesannwyn -p 1 -b 16 -s 8 Map_Dump.esp

Remember the image dimensions (5888x3392) and the bottom left coordinates (-120,-14) of the image! Example data provided for SHOTN.

EDITING THE HEIGHTMAP
The 16-bit RAW file can be opened in Photoshop. Just input the image size (5888x3392), bit depth (16 bits), and byte order (IBM PC). We only have 1 channel, header size is 0.
There are three constraints for the heightmap to keep in mind before re-importing:
1) Ocean floor should be at 0 (#000000)
2) Minimum land should be at 2 (#020202)
3) Maximum height should be at 127 (#7F7F7F)
Then, open the map in your preferred image editor, and:
1) Create a new layer on top with solid fill #020202.
2) Merge down on the heightmap with linear dodge (add)
3) Paint over any mountain peak that's brighter than 127.
4) Select all ocean pixels and replace them with transparency.
Optional: Create a pure white background layer for better visibility.
5) Make your edits to the heightmap. To raise land, paint with #020202. Avoid areas with high gradients (going from white to black within a fex pixels on land). Avoid visible edges in the heightmap. Avoid using the same color too much along the edges of the map.
6) Create a new layer on the bottom with solid fill #000000
7) (This is the point of no return, so make sure to make a backup .psd of the file!) Merge the heightmap down on it.
8) Add 2px gaussian blur.
9) Make sure the image is flipped vertically. This is due to how raw image data is interpreted.
10) Save as "tesannwyn-import.raw", Photoshop RAW format, Interleaved channel order, IBM PC byte order.

==================================
===                    IMPORT                    ===
==================================
IMPORTING THE HEIGHTMAP
Use the following command with the example data filled in:

tesannwyn -i Morrowind -p 1 -b 16 -d 5888x3392 -x -120 -y -14 -h -256 -s 0.125 -o -8192-0 tesannwyn-import.raw

Explanation:
-p 1 This is a RAW file.
-b 16 This is a 16-bit file.
-d 5888x3392 These are the image dimensions.
-x -120 -y -16 These are the bottom left coordinates.
-h -256 This is the offset from z=0. This means that our ocean floor (#000000) is actually going to be under water.
-s 0.125 This is the scaling value. More on this later.
-o -8192-0 This removes any cells that don't contain meaningful height data (e.g. ocean bottom wilderness).

SCALING
Photoshop's 16-bit mode actually exports to a 15-bit grayscale image. This means that instead of 65536, our maximum data value is only 32768. Again, we need to take the maximum allowed height in the TES3 engine (z = 32768, THU = 32768/8 = 4096), and divide the two numbers: 4096/32768 = 1/8

DEBUGGING
You might see errors popping up for underflows/overflows. You can fix this by making a new heightmap export and looking for bugs in your height data:

tesannwyn -p 2 -b 16 tesannwyn.esp

This is a 16-bit image, which you can open in any image editor and look at the individual color channels. Only look at the red channel, which is just a black-and-white map of your landmass. If you see any white where there shouldn't be (unwanted lakes on land) or any black where there shouldn't be (unwanted land in the ocean), this means your heightmap is broken on those spots. Try painting over the landmass there with some other color to fix the issue.
If you see old heightmap exports/imports getting burned into everything you do, try deleting the tesannwyn temp folder.

USING YOUR IMPORTED LANDMASS
Open the imported .esp in the Construction Set as active file and click "save". You might see error correction messages popping up, just click yes (or go back to the debugging section).

CUTTING UP A LARGE LANDMASS
With tes3cmd, it's rather easy to cut up land records that are over 250 MB (a file that size would make the CS crash). Just make two copies of your huge ESP, and name them "Tamriel1.esp" and "Tamriel2.esp". Then, using regex, you can specify where to cut the cells. The ^\( and ^\(- parts in the beginning are very important, as they define the beginning characters of an exterior cell record.

tes3cmd delete --id "^\(-(8[0-9]|9[0-9]|[12][0-9]{2}|300)," Tamriel1.esp
tes3cmd delete --id "^\(-([0-9]|[1-6][0-9]|7[0-9])," Tamriel2.esp
tes3cmd delete --id "^\(([0-9]|[1-8][0-9]|9[0-9]|[12][0-9]{2}|300)," Tamriel2.esp

The intervals were generated using this tool: http://gamon.webfactional.com/regexnumericrangegenerator/