Tamriel Data

111 posts / 0 new
Last post
worsas's picture
worsas
Developer Emeritus
Joined:
2016-01-26 12:34
Last seen:
7 months 1 week ago

Opening this dedicated data thread with a new version of the translation file.

I fixed all of the errors pointed out by Seneca and many more I discovered myself. But there are a number of changes, too.

After some forth and back, I figured it would be best to change the order of the province-race- prefix for the container ids, so the province comes infront of the race designation. In the end it will be more practical in most cases. So it is T_MwCom_Var_CrateLong_Empty instead of T_ComMw_Var_CrateLong_Empty now. Like that all relevant containers for TR and the other projects respectively are kept tight together.

Another thing I did is doing away with glb- levelled items and – creatures and simply assigning them to the province of the project they are from. I couldn’t really find a good system for reliably saying, if a levelled creature or -item should be glb or must stay mw or sky or cyr and it ultimately doesn’t matter too much, I guess. The levelled creatures and items are mainly suited for the needs of the individual project, which is a good reason for leaving them province-specific.

AttachmentSizeDate
Plain text icon Translation.txt730.33 KB2016-05-22 21:26
Seneca37's picture
Seneca37
Developer EmeritusDeveloperExterior DeveloperInterior DeveloperReviewer
Joined:
2015-07-05 20:55
Last seen:
11 months 1 week ago

Here are a couple of things to think about.
1) How will we name similar items that appear in different catagories. For example, TR likes to have both static and misc vases. Static vases are used when a modders uses a container plant. Misc vases are used with ingredients. We also have some generic Static cave pieces, and then the same piece in Activators that is somehow special (usually a sound attached to it).

2) Down the road when everything is merged we’ll need to update uesp.wiki. How is this going to be done? Will all this stuff be under Tamriel Rebuilt, or would a new cross project site need to be set up. I’d like to see the spells and enchantments listed there for ease of use for the modders.

worsas's picture
worsas
Developer Emeritus
Joined:
2016-01-26 12:34
Last seen:
7 months 1 week ago

1) How will we name similar items that appear in different catagories. For example, TR likes to have both static and misc vases. Static vases are used when a modders uses a container plant. Misc vases are used with ingredients. We also have some generic Static cave pieces, and then the same piece in Activators that is somehow special (usually a sound attached to it).

The way I have been going about objects like these was mostly adding St or Stat to the name of the static version or the other way around act, if the first version was a static and the second version an activator. T_Com_SetHarbor_Boat_01 → T_Com_SetHarbor_BoatAct_01, for example. In my opinion it’s good enough like that because these intersections are usually not that frequent.

On the uesp-stuff.. In my opinion, it would be fine to handle all objects in the Tamriel Rebuilt section and maybe add a note, if something is from PT_Data.bsa.

Seneca37's picture
Seneca37
Developer EmeritusDeveloperExterior DeveloperInterior DeveloperReviewer
Joined:
2015-07-05 20:55
Last seen:
11 months 1 week ago

Below is the latest spreadsheet version

AttachmentSizeDate
File Translation_4B.ods1.68 MB2016-05-28 19:43
Seneca37's picture
Seneca37
Developer EmeritusDeveloperExterior DeveloperInterior DeveloperReviewer
Joined:
2015-07-05 20:55
Last seen:
11 months 1 week ago

The use of Set in the designations is unclear to me. Even though the Set… designator keeps groups of objects together, you will never use items from various sets. For example, T_Nor_SetFarm_… should not be used with T_Nor_SetHarbor_…. Why not just call them T_Nor_Farm, and T_Nor_Harbor. And anything with just _Set_, make that a _Var_.
 

worsas's picture
worsas
Developer Emeritus
Joined:
2016-01-26 12:34
Last seen:
7 months 1 week ago

Set refers to objects used for the building of settlements. Nor_Set_, for example, contains things like windows, doors, chimneys, way signs, etc. Putting all of those into var would just spread them between random objects that have no place in any of the other categories. it is practical to have the common-purpose set-set in proximity of the setFarm-set, as the farm houses use windows and doors of the former.

Legion and Highway are for legibility sake the only exceptions at this point. Apart from that you will find all buildings/structures either behind set or behind dng. There is also some intersection between dng/set and furn, because settlement sets sometimes include their specific furniture.

You will never get an absolutely perfect system for this kind of thing. We could take steps to make things more consistent in some places, but it would probably make ids less intuitive at the same time. But set is definitely a stayer, if you ask me.

Seneca37's picture
Seneca37
Developer EmeritusDeveloperExterior DeveloperInterior DeveloperReviewer
Joined:
2015-07-05 20:55
Last seen:
11 months 1 week ago

Ok, how you are using Set (aka settlement) seems perfectly logical to me. I had been reading as a group of similar objects.

Looking over the translations, I see a bunch of Items that need to be deleted (in TR’s case a bunch of books). I beleive the reason these were never deleted from DATA is the fact that they would break the players game. Since we need to update a players save game anyways due to the new IDs, now would be a good time to remove them permanently from DATA a the player's .ess file.
 

Seneca37's picture
Seneca37
Developer EmeritusDeveloperExterior DeveloperInterior DeveloperReviewer
Joined:
2015-07-05 20:55
Last seen:
11 months 1 week ago

Sorry for taking so long to reply again. It took longer than anticipated to create the region map.



We'll need region maps to correlate the region designations used to the specific areas. Below is the current Region map for Morrowind as of v1603(ish). I added our Southern regions to the map that were not in v1603. Unfortunately, our region names in TR Preview may change. I’m not sure on how set the region names are for the other groups.



A couple of points

1) We do not have a UV region. I think at some point there was going to be a Upper Velothi Mountain region, but it is currently called Northern Velothi Mountains.

2) We do not a GB region. That designation needs to be removed. I believe that designation was carried over from the original designers filenames.


 

worsas's picture
worsas
Developer Emeritus
Joined:
2016-01-26 12:34
Last seen:
7 months 1 week ago

So TR will scrap Uld Vraech? At least in this thread here, it is still planned as an independent region:
http://tamriel-rebuilt.org/forum/velothis-landscapes

I’m missing any information saying that TR will discard their plans for that region. And I’m not sure what you mean with a GB-region. I haven’t used GB anywhere in my ids, as far as I’m aware.

Atrayonis's picture
Atrayonis
Lead DeveloperDeveloperInterior DeveloperQuest Developer
Joined:
2015-09-28 20:13
Last seen:
14 hours 55 min ago

This might be a good moment to raise the spectre of actually deciding TR’s regions once and for all again. I have several mutually exlusive region maps of TR and until Seneca made this one, there was no really “official” one which reconciled the southern regions with the existing one.

Perhaps another thread? I do have that issue of the Velothi/Valus mountains still half-sitting on my harddrive (Gnomey wrote it and made some pictures, but it’s incomplete and a tangent from the discussion we had on IRC with – I think – Scamp).

10Kaziem's picture
10Kaziem
Lead DeveloperDeveloperInterior Developer
Joined:
2015-12-12 23:47
Last seen:
6 days 15 hours ago

Raise it at the meeting on Saturday. (And it could use its own thread, yes)

Does: concepts, textures, youtube vids, admin stuff e.g. PR, handbook, assets, small website things. Activity level: wildly unpredictable. Still active. Find me on Discord.

worsas's picture
worsas
Developer Emeritus
Joined:
2016-01-26 12:34
Last seen:
7 months 1 week ago

Here is yet another version of the Translation-file including translations for all objects added in the recent updates of Skyrim_Data and PC_Data. Moreover it also includes ids for sounds, classes, factions, scripts and groundtextures, which were missing previously. Also fixed a few translations for TR-objects that got messed up in between.

To speak the truth, I would like to take some distance from modding in the coming months and I hope we can pull out this data move before that, of course, only as long as it is still desired.

I’ll also write up the rules for these ids in more detail over the coming weekend.

Edit: The ids for the new objects in the two PT-data files are not neatly ordered like the others at this point. But in my opinion, we should actually disregard the excel files, because we cannot use them for the patching program anyway. I’d rather like to set up some kind of demo-data.esm for people to view, which should give a better idea of how these object ids will be like in practice. I should be able to write a program that can setup a preliminary data version.

AttachmentSizeDate
File Translation.7z89.12 KB2016-07-03 21:50
worsas's picture
worsas
Developer Emeritus
Joined:
2016-01-26 12:34
Last seen:
7 months 1 week ago

Hello again,

would it serve the project, if I created a preliminary version of Tamriel-Data, foremostly intended for the sake of viewing the contents and the lists so everyone gets a more concrete picture of it all?

A few weeks ago I wrote some java code for the skyrim project to auto-fix instances of certain objects, which can read and interpret esm/esp-files and iterate through the contents. Using this as a base, i should be able to write something that can set up a prelimary data version with the help of the translation.txt.

seneca was originally going to take care of the programming part, though, but he has not been around much recently and it also seems like we misunderstood each other on a number of points. I’m currently not sure how, when or even if this move will take place. I’m, kind of, panicking about it, because the writing of the translation.txt has taken me really many hours of manual judging and writing work, looking back and forth between cs and the input mask for the new id. And now we are acually just a stone’s throw again from having a working new data.

So, please, I really need some reply to this topic. Otherwise I’m going to feel really stupid about the whole thing, certainly more than I already do.

Atrayonis's picture
Atrayonis
Lead DeveloperDeveloperInterior DeveloperQuest Developer
Joined:
2015-09-28 20:13
Last seen:
14 hours 55 min ago

The lack of feedback is not because nobody cares, and it would most certainly help.

More drastically said: this is the only way this can still be implemented.

Seneca has recently announced that he will not be consistently active for the forseeable future, so there’s pretty much nobody on TR’s end anymore who can check back against your proposals, much less (automatically) implement them. If you don’t do it, nobody else can anytime soon.

worsas's picture
worsas
Developer Emeritus
Joined:
2016-01-26 12:34
Last seen:
7 months 1 week ago

Ok, thanks. I’ll see about the needed programming then. So far it’s been actually enjoyable to interpret and abstract the esp-format within java.

Non-responding on a forum can really drive me mad, btw. You blocked me on facebook and now you are going to die… sort of.

Atrayonis's picture
Atrayonis
Lead DeveloperDeveloperInterior DeveloperQuest Developer
Joined:
2015-09-28 20:13
Last seen:
14 hours 55 min ago

I’m not on facebook.

If you meant the official Tamriel Rebuilt facebook, I haven’t logged into that ever. I’m not sure who has blocked you and why.

worsas's picture
worsas
Developer Emeritus
Joined:
2016-01-26 12:34
Last seen:
7 months 1 week ago

Err, no, that facebook stuff I said was a reference to a silly dubstep song from years ago. Nothing that has to do with reality.

I rarely go on facebook myself and I have never visited the TR- facebook account.smiley No worries. I will try to avoid mistakable statement in future.
Edit: And, of course, I’m not after anyones life.

10Kaziem's picture
10Kaziem
Lead DeveloperDeveloperInterior Developer
Joined:
2015-12-12 23:47
Last seen:
6 days 15 hours ago

Hi Worsas, yes, please do; with Seneca being inactive for a while it’s probably up to me to review this sort of thing as I was the other person who worked on our renaming deal last year. I’m busy at the moment but would definitely love to see an updated TR data to look through.

Does: concepts, textures, youtube vids, admin stuff e.g. PR, handbook, assets, small website things. Activity level: wildly unpredictable. Still active. Find me on Discord.

worsas's picture
worsas
Developer Emeritus
Joined:
2016-01-26 12:34
Last seen:
7 months 1 week ago

In the meantime I have worked on some kind of abstraction layer that treats file contents as rows in tables, very much dislodged from how the records in the file are actually stored. When I have set up all of the needed tables (right now there is only cell_objects) creating the code that sets up the data files or patches peoples content files and savegames will be very easy.

We could use this functionality to replace certain plants in the Thirr river region in one go without manual search&replace in each cell, for example. See what I did in the first screenshot with one working fle at SHOTN. The table and column names are abit inconsistent right now.
https://dl.dropboxusercontent.com/u/22362022/pics/quickeditor1.png
https://dl.dropboxusercontent.com/u/22362022/pics/quickeditor2.png

10Kaziem's picture
10Kaziem
Lead DeveloperDeveloperInterior Developer
Joined:
2015-12-12 23:47
Last seen:
6 days 15 hours ago

That sounds amazing and beautiful.

Does: concepts, textures, youtube vids, admin stuff e.g. PR, handbook, assets, small website things. Activity level: wildly unpredictable. Still active. Find me on Discord.

worsas's picture
worsas
Developer Emeritus
Joined:
2016-01-26 12:34
Last seen:
7 months 1 week ago

As promised here is a test version of the Tamriel_Data-Files. I also included the latest version of TR_Data.bsa I had, so the people on the PT-board won’t have to download it separately.
https://drive.google.com/file/d/0B06OJzIcHVkyOE15ZFA5VWpaVTg/view?usp=sharing

I pretty much spent the whole weekend and most of this day on writing and testing and testing the program you can use to patch working files and savegames. At this point everything seems to work well, except that the byteCode of scripts doesn’t get automatically updated, so if you patch TR_Mainland.esm, for example, you will have to go through every script in the CS after and press save once to regenerate it, but the ids in the scripttexts get updated already. This issue shouldn’t affect patched savegames, as script instances in savegames don’t contain code as far as I have seen.

And at this point you will have to have java on your pc to run the patching program, like with the esp-quickeditor. I’ll see, if I can turn this into a native executive in the coming days or weeks. That is, if we come to a agreement of using this new data setup. You would have to communicate the need of patching of their savegames to people and adding PT_Data.bsa to their archive-list (or the program could addtionally add this entry to the Morrowind.ini)

Atrayonis's picture
Atrayonis
Lead DeveloperDeveloperInterior DeveloperQuest Developer
Joined:
2015-09-28 20:13
Last seen:
14 hours 55 min ago

Well, I’ve only clicked through the assets so far, but I’m very very happy with how this turned out.

The naming seems logical and pretty intuitive. I had no problem identifying what I was adding and in what context it should be used.

The work with regenerating the scripts will just take some time, I guess, but that should be manageable.

I’m going to put it on the agenda for this week’s meeting – I think we should move ahead boldly and see if we can’t put the new structure into TR1609, along with the regions and city renames. That way, soon as we start adding more of the pending assets we already have the permanent solution names.

The BSAs are a never-ending source of problems already. I don’t see how adding two instead of one will create more confusion than it already does (if the user in question didn’t already have both TR_bsa and PT_bsa from using both individual projects), so I think we’re good with changing the bsareg thing we ship with and announcing it in bright, big letters.

Rats's picture
Rats
Lead DeveloperDeveloperExterior DeveloperInterior DeveloperQuest DeveloperReviewer
Joined:
2014-01-08 21:55
Last seen:
1 day 47 min ago

So… many… new… resources… It’s clear that tremendous effort has been put into this. Kudos to all of you involved!

Ragox's picture
Ragox
Senior DeveloperDeveloperInterior DeveloperReviewer
Joined:
2016-01-17 01:30
Last seen:
1 month 1 week ago

This is truly amazing, it must’ve been so much work to organize it all!
Also with this we’ll finally have complete Dragonscale, Newtscale, Studded Leather and Trollbone armor sets among countless other goodies laugh

Atrayonis's picture
Atrayonis
Lead DeveloperDeveloperInterior DeveloperQuest Developer
Joined:
2015-09-28 20:13
Last seen:
14 hours 55 min ago

I’ve been running the patcher over the current TR_Mainland, TR_Preview and TR_Travels. Haven’t tried TR_InDev yet, but that mainly differs with some bug fixes and the newest heightmap. I’m not done with this, but I’d like to toss my findings so far.

First, most of it works admirably well.
However, I ran into the following issues on TR_Mainland:

TR_m1_q_IC_3_Mora_Gindal
Line 41: Could not find variable or function T_IngFood_BreadColovianMw_01b

This is apparently because the old item is "tr_ingred_bread_01b" and the translation file has:
tr_ingred_bread_01:T_IngFood_BreadColovianMw_01
tr_ingred_bread_01b:T_IngFood_BreadColovianMw_02

Simply switching those entries around might be enough (I corrected them in TR_Mainland manually for now).

TR_m2_q_29_OlmuTravel
Line 25: Object/ScriptID "T_De_Set_BannerSaintAralor_01_refu"

This used to be "TR_act_banner_aralor01_refu" - and it still is, since this activator is defined in TR_Mainland not Tamriel_Data.
I don't know why, but I assume that banners were first defined in TR_Mainland and then eventually moved to TR_Data. This apparently didn't happen here and should be done.

TR_m2_q_4SimuindScript is fucked up.
Same with TR_m2_q_4_TanyelieScript.

 

It appears to happen because those scripts start with a line break instead of with the the "Begin" line.
While that needs to be corrected before the patcher is run, it's the most serious bug I've seen so far.

And the following with TR_Preview:

T_ScNpc_Mw_Map3
Miss mached quotes
I don’t know where that comes from. It happens when I join the Mages Guild in Old Ebonheart.
This happens because the command was incorrectly transitioned. Its now PCJoinFaction "T_Mw_MagesGuild with the last quote missing. This seems to happen with every dialogue result involving a mainland guild – the very last letter (almost always a quote) is missing.

And the following with TR_Travels:
Pretty much breaks the exterior cell edits. These will need to be redone. Since this happens every time, it’s no big deal.

There’s a couple of things we need to do before at the very least before we can get this working (hopefully with TR1609):

  1. Move more scripts and items to Tamriel_Data.
  2. Implement the region changes.
  3. Fix the Grille mesh placement, as the recent mesh fix has made them all misaligned.

Since they are too big to attach, I’ve uploaded the transitioned files here.

Ragox's picture
Ragox
Senior DeveloperDeveloperInterior DeveloperReviewer
Joined:
2016-01-17 01:30
Last seen:
1 month 1 week ago

The archive you posted is corrupted, but it really sounds great, seems like the amount of bugfixing will be manageable.

Atrayonis's picture
Atrayonis
Lead DeveloperDeveloperInterior DeveloperQuest Developer
Joined:
2015-09-28 20:13
Last seen:
14 hours 55 min ago

Mistake on my end, file reuploaded.

Gnomey's picture
Gnomey
Lead DeveloperDeveloperExterior DeveloperInterior Developer
Joined:
2015-08-10 20:50
Last seen:
19 hours 1 min ago

I’ve only been able to get around to looking at the Tamriel_Data preview this evening, mostly just skimming through the selection, and so haven’t been able to form a strong impression yet. The rough impression I do have so far is very positive though; by and large everything seemed to be grouped intuitively. The only things that gave me pause were that the drinks were mixed with the potions (ie.  T_Nor_Drink and T_Nor_Potion instead of T_Drink_Nor and T_Potion_Nor or something) and that the harbour assets were scattered in the same way. Not sure if that will be an issue though; as I say I haven’t really formed a strong impression yet.

Great work on this, and I certainly already think it’s leaped well over the admittedly low hurdle of being more intuitive than TR_Data!

worsas's picture
worsas
Developer Emeritus
Joined:
2016-01-26 12:34
Last seen:
7 months 1 week ago

Thanks for giving the patcher and the data files a test. I’ll see, if I can diminish those issues.

The issue with the missing quote in the resulttexts is that the patcher probably crops the last character from the text at this point. Usually all strings in the esp-file are null-terminated and the program internally removes the last letter for the internal handling and adds it again for the export, but in case of some value fields there does not seem to be this null-termination and removing the last letter crops a part of the text. I have not been smart enough to check whether the cropped letter is actually the null-terminator previously. Sorry for that.

I and seneca have been discussing the need of adding additional reference interior cells to this data to give interior modders an idea of which objects they can use to retain consistency with previously made interiors. Though, after speaking to anonytroll about it again, it probably won’t be needed for now. But we can still take it into consideration later.

Dialogue and regions within Tamriel_Data should actually be moved to TR_Mainland.esm. Objects on the other hand should be in the Data Files.
Edit: Unless they are unique to interiors or quest-related, in which case the naming of the object should make it clear that it is such an object. TR_act_banner_aralor01_refu should probably get renamed to TR_m2_bannerAralor_refu or something similar, as it contains a quest-related script. We should probably add translation lines for these objects, too and put them atop of the translation.txt.

The biggest worry I have had with these Data Files is that TR might find their own assets to be mal-placed between much stuff from SHOTN and P:C. De And Mw both happen to be in between the range of letters, you can neither scroll to the beginning nor to the end to get to your stuff.

 

Atrayonis's picture
Atrayonis
Lead DeveloperDeveloperInterior DeveloperQuest Developer
Joined:
2015-09-28 20:13
Last seen:
14 hours 55 min ago

The null-terminator is a problem. I think we can just fix that by hand, it will just a day or two. I’m more concerned about mod authors or users (when the authors have disappeared) running these patchers and then encountering these issues.

I’m not too keen on moving regions off Tamriel_Data, though.
I would prefer a unified region structure, it would lessen confusion, force us to find abbreviations that don’t conflict with each other (also relevant for the object names), and it would be really helpful for mods which create islands to have ocean regions available to them without dragging in the more swiftly-changing content of worldspace edits.
It would also be really inconvenient to have to sync regions between TR_Mainland and TR_Preview (the latter which doesn’t have the former as a master).

Moving dialogue should not be an issue. What about scripts?
 

10Kaziem's picture
10Kaziem
Lead DeveloperDeveloperInterior Developer
Joined:
2015-12-12 23:47
Last seen:
6 days 15 hours ago

Regions should be in TR_Data, I think, though I can see scripts/dialogue being elsewhere. I’d want input from the people who work with dialogue on that though.

Does: concepts, textures, youtube vids, admin stuff e.g. PR, handbook, assets, small website things. Activity level: wildly unpredictable. Still active. Find me on Discord.

Pages