TL;DR
Teach modders to write changelogs in such a way that they make best use of Factorio's capabilities.Why?
There are lots of mods around, and most of them have changelogs. It is good to have changelogs (you want to know before updating whether an update may break something) -- it is even better if one can read them directly in the game and doesn't have to switch to the browser and look in the mod portal. However, there are many mods where the "Changelog" button in the game's mods manager is inactive. Sometimes this happens because of a real bug (e.g. typos), sometimes the authors never really cared enough to fix such seemingly unimportant errors, and sometimes they just hadn't realized that changelog files require a special syntax.What?
Recently, I happened to start Factorio from the terminal instead of using the shortcut in the menu. I was updating some mods and checking out others by selecting them in the game's mods manager. When the game restarted, I saw the output that is usually hidden and noticed a number of lines like these:Code: Select all
2944.131 Error CachedChangelog.cpp:37: Failed to parse changelog for mod [insert name here]: invalid changelog file, error on line 1.
In short: rules and example of a working changelog file
As suggested by GrumpyJoe, here is an example of a working changelog file (snippet from Schall Alien Mutation, with some changes to categories):Code: Select all
---------------------------------------------------------------------------------------------------
Version: 0.17.2
Date: 2019.03.21
Balancing:
- Updated pollution_to_join_attack values to base game 0.17.12 standard.
Optimisations:
- Code tidying and cleanup.
---------------------------------------------------------------------------------------------------
Version: 0.17.1
Date: 2019.03.07
Changes:
- Updated thumbnail for 0.17.
---------------------------------------------------------------------------------------------------
Version: 0.17.0
Date: 2019.03.01
Info:
- Updated to 0.17.
---------------------------------------------------------------------------------------------------
Version: 0.16.3
Date: 2019.02.07
Optimisations:
- Optimized code to make better order in spawn tables.
Graphics:
- Added icons to alien types, in case any mods are using them.
---------------------------------------------------------------------------------------------------
Version: 0.16.2
Date: 2018.12.15
Locale:
- Russian is available.
---------------------------------------------------------------------------------------------------
Version: 0.16.1
Date: 2018.12.06
Optimisations:
- Internal name changes.
Locale:
- Simplified Chinese is available.
---------------------------------------------------------------------------------------------------
Version: 0.16.0
Date: 2018.11.25
Info:
- Initial release
Locale:
- English, German, traditional Chinese are available.
License:
- Using Bob's Copyright License. Document included.
- Never use the tab character (\t) anywhere in the changelog file -- it's an illegal character!
- The changelog entry for each new version of your mod must begin with a header line. It must contain exactly 99 dashes ("-"), nothing else, no spaces anywhere!
- Immediately after the header line, include a unique version number. Don't indent the line! The required format is "Version:*X.Y.Z", where X and Y must be (unsigned, positive) integers while Z may be any character. Z will be translated to '0' if it's not an unsigned number, characters after a number will be truncated.
- The release date is optional. If you include it, don't indent the line! The required format is "Date:*[date string]". The date string must not be empty -- if you don't know the date, just leave out the entire line!
- Add categories! Category lines must be indented with 2 spaces. They must end with a colon, no spaces are allowed after the colon! Category names may contain spaces. The required format of such a line is "**My category:". Here is a list of the "official" categories used by the developers for Factorio's changelog.
- Under each category, add at least one list entry. These lines must be indented with 4 spaces and begin with "-", followed by a space. One entry can span several lines. Each additional line must be indented with 6 spaces. The required format is "****-*[text]" or "******[continued text]".
- You may insert empty lines before a category line and at the end of a changelog entry. Empty lines must be really empty -- they must not contain any spaces!
- The file must be named "changelog.txt". It must be in the root directory of your mod, and it may be encoded as UTF8 with or without BOM!
Schallfalke has a better explanation. He observed that if a mod was initially released, the changelog wouldn't be shown on the modportal and it would only be shown in the game's mod manager after it had been installed. If an update of a mod was uploaded and that update contained a changelog, this changelog would be displayed on the mod portal and (if it was a valid changelog file) in the game's mod manager for both installed and new mods.
Getting started … detailed explanation
First thing you should be aware of is that your changelog has to follow some rules regarding syntax and formatting in order to be recognized by the game's parser. Consider this example:Code: Select all
Version 0.4.1: Updated for Factorio 0.17. Fixed conflict with nanobots.
Version 0.3.2: Fixed ghost variation offset.
Version 0.3.1: Updated for Factorio 0.16.x. Decreased file size. Added a button to deselect the input field.
[…]
So, how about this?
Code: Select all
1.0.1:
- Updated mod to work with 0.17.x
---------------------------------------------------------------------------------------------------
1.0.0:
- After some hours of testing I declared this version v1.0.0; I'm quite happy with how it is now.
---------------------------------------------------------------------------------------------------
0.6.0:
- Fixed the balancing
- Upgraded the graphics to 64x64 tiles instead of the old 32x32
---------------------------------------------------------------------------------------------------
0.5.1:
- Added the Github page as the homepage URL in the info.json
So, what syntax does Factorio expect?
Syntax rules
Let me start with a disclaimer: I have no clue! I am just a normal player, not a modder, and I couldn't find a guide to changelog syntax. All information I provide comes from comparing changelog files from mods I found a parsing error for in the output with changelog files from mods that could be parsed successfully.In short, what I've written here boils down to more or less qualified guesswork. That means the information and explanations I can give are likely to be either wrong or incomplete. This posting can only a be starting point, making modders aware that their changelog files may need some tweaking. I would appreciate it very much if anybody with more knowledge on this matter -- modders or even devs -- would correct me where I've been wrong, or add more information. Perhaps there even is a document laying out the rules for changelog files that can be parsed by the game? If so, please link to it!
Now, on to the real stuff …
- Structure of a changelog file (I use the -- working -- file from Schall Lamp Contrast as an example here)
- The changelog file is a normal text file containing one or more changelog entries.
Update (2019-03-08): It should be named "changelog.txt" and be stored right in the root directory of your mod.
Update (2019-03-12): According to steinio, it may be required that the changelog file is encoded as UTF-8 without Byte Order Mark (BOM).
Update (2019-03-23): Bilka confirmed that changelog files encoded as UTF-8-BOM will break parsing. So, please encode your changelog files in plain UTF-8, without BOM!
Update (2019-09-21): According to Bilka, changelog files now may be encoded as UTF8 with BOM.
- The changelog file is a normal text file containing one or more changelog entries.
- Structure of a changelog entry
- A changelog entry starts with a header line. If the parser encounters such a line, it will look for a new changelog entry in the
following linesnext line.Code: Select all
---------------------------------------------------------------------------------------------------
- A header line contains only dashes ("-").
I guess, technically it would be enough if the first character is a dash; then again, humans can make more sense of it if the header lines span the whole line width.
Implications: This means thatthe first linethe first non-empty line of the changelog file must be a line with only dashes. It also means that the last non-empty line of the changelog file must not be a line containing only dashes -- otherwise the parser will throw an error because it expects to find another changelog entry. (I mention this explicitely because these seem to be the most common mistakes causing the parser to abort with an error message. Apparently, many modders consider a line consisting of only dashes to be a separator line, so they omit it at the start of their file. Also, it seems some modders want to make their changelog file look neat, so they use dashed lines as top and bottom borders.)
Update (2019-03-19): It is now official that the header line must be the first non-empty line in the changelog file. That means it doesn't matter if you hit ENTER a couple of times before starting the first header line. However, keep in mind that a line containing only space characters does not qualify as an empty line!
Update (2019-03-29): According go badtouchatr, the header lines must consist of exactly 99 dashes. - Update (2019-12-30): After the header line, add a version field:
Code: Select all
--------------------------------------------------------------------------------------------------- Version: 0.17.0
- "Version: …" must be at column 1 of the line, there must be no spaces preceding it! Also, the version line must immediately follow the header line -- no empty line is allowed between them!
- The version number must have the format X.Y.Z. X and Y must be unsigned (positive) integers, while Z may be any character (except tabulators). If Z is not an unsigned integer, it will be translated to '0', so
means the same as
Code: Select all
--------------------------------------------------------------------------------------------------- Version: 0.17.0
orCode: Select all
--------------------------------------------------------------------------------------------------- Version: 0.17.x
If Z is an unsigned (positive) integer followed by another character or string, the string will be truncated by the parser. Thus,Code: Select all
--------------------------------------------------------------------------------------------------- Version: 0.17.-1
means the same as, for example,Code: Select all
--------------------------------------------------------------------------------------------------- Version: 0.17.0
Code: Select all
--------------------------------------------------------------------------------------------------- Version: 0.17.0-beta
- Duplicate versions will result in a parsing error. However, recent tests have shown that it's not easy to tell unique versions apart from duplicate versions.
WILL NOT WORKThis will workIn the working examples, the output will probably look different than you expect. Separate changelog entries with the same version number will not be displayed separately (as they appear in the changelog file), they will be merged instead. Thus, the output for version 0.0.4 from the example will look like this:Code: Select all
--------------------------------------------------------------------------------------------------- Version: 0.0.4 Date: 2019-12-29 Test: - Same version, one without date, the other with date will work. - See above!
- Even if you have used two changelog entries with the same version number and these versions are not regarded as duplicates, there is room for an error:
In this case, you will get an error like "line X is a duplicate of line Y" exactly because these two blocks would be merged. So, the best advice is to avoid any situation where duplicate versions may be an issue!
Code: Select all
--------------------------------------------------------------------------------------------------- Version: 0.0.5 Test: - Same version, one without date, the other with date won't work with identical item lines. --------------------------------------------------------------------------------------------------- Version: 0.0.5 Date: 2019-12-29 Test: - Same version, one without date, the other with date won't work with identical item lines.
- The version number may not be "0.0.0" -- this will result in a "missing version" error!
- If only little has changed between two versions -- e.g. if you updated your mod to the latest version of Factorio in 0.17.0 and decided to add a changelog in 0.17.1 after reading this tutorial -- it seems to make sense to use the latest version number. So, instead of
I'd recommend using
Code: Select all
Version: 0.17.0/0.17.1
because that was the version where the last change documented in this block has been implemented.Code: Select all
Version: 0.17.1
- Add a date field (optional):
Update (2019-03-19): Please note that "Date: …" must start at column 1 of the line, there must be no spaces preceding it! There may be empty lines before or after it.
Code: Select all
--------------------------------------------------------------------------------------------------- Version: 0.17.0 Date: 2019.03.01
I'm not sure whether the date field is actually required by the game's parser.However, it is nice to know when a mod has been updated by the author, so just include it, please! By the way, it shouldn't matter to the parser whether you write "2019.03.01", "2019-03-01", "01.03.2019", or whatever. I take it that everything after "Date:" is just interpreted as a string by the parser.
Update (2019-03-08): As Blue Templar pointed out, the date format does matter to humans! Factorio is played around the world, and people in different countries are used to interpret date strings differently. "08/03/2019" could be read as "March 8, 2019", but also as "August 3, 2019". So Blue Templar suggests that the value of the date field should be given in Coordinated Universal Time (UTC), which I strongly support as having all mods using one time format makes things unambigious. A date string with the current time in UTC notation looks like this:For more information about this format, please refer to RFC 3339.Code: Select all
Date: 2019-03-08 21:49:16+01:00
Update (2019-03-19): According to my tests, the date field is optional. However, if you set it, it must not be empty! The contents of the date field must be a non-empty string -- ideally the current date and time (UTC), but something like "November 2018" would also be possible. If you don't remember when you have released a version, you could just insert a space after the colon ("Date: "). In this case, you could also leave out that line altogether: the game's parser will then kindly insert "Date: " on its own.
Important: Take care to use unique version numbers and dates! I've seen one mod where the parser complained about a duplicate date field. Actually, the author had just copied the version and date fields, so there were two entries with the same version/date string in the changelog file.(I would have expected the parser to stumble over the duplicate version field, which should have been parsed first -- don't know why it didn't.)If you release more than one version on the same day, I'd suggest to add the time to the datefield, like this:
Update (2019-03-08): Time format changed to UTC, as suggested by Blue Templar.Code: Select all
Date: 2019-03-01 11:36:25+01:00
Update (2019-09-21): Bilka leaked secret information! A "Date:" line will only be recognized if there is a space after the colon, but if you are using an editor that cuts off trailing spaces at the end of a line, it will be regarded as a normal changelog entry and cause an error because it is not properly indented.You can prevent this by either not including the "Date: " line, or by adding at least one question mark:Code: Select all
Failed to parse changelog for mod xxx: invalid changelog file, error on line 41, line does not start with exactly ' - ' or exactly ' '.
The space between colon and question mark is important, but it really doesn't matter how many question marks you use -- I've tried 1, 2, 3, and even 300 without getting an error.Code: Select all
--------------------------------------------------------------------------------------------------- Version: 0.17.4 Date: ?
- So far, we have just defined the header: There is a line signifying the beginning of a new entry, there is the version number, and there is the date of when the changes have been implemented. Now it's time for the content! Just add a description of your changes,
optionally [spoiler](?)[/spoiler] embedding them inby adding other fields:
That's it!Code: Select all
--------------------------------------------------------------------------------------------------- Version: 0.17.0 Date: 2019.03.01 Features: - Updated to 0.17.
Update (2019-03-08): According to my tests, all information must be provided as contents of a field. Here is an example that doesn't work:Code: Select all
--------------------------------------------------------------------------------------------------- Version: 0.17.0 Date: 2019.03.01 Added magnificent new features!
- A changelog entry starts with a header line. If the parser encounters such a line, it will look for a new changelog entry in the
- Order of changelog entries
I think in all of the changelogs I've seen, new entries have been added to the top of the file. However, the in-game changelog viewer shows the oldest changelog entry at the top and the latest at the bottom. I therefore believe that order doesn't really matter. Insert new entries at the top of the file if you feel like it, or add them at the bottom if that is more to your taste. I guess you could even put them in arbitrary order if you really are into stuff like that! It should make no difference as far as the game's parser is concerned, but it could drive you and/or your mod's users into insanity. - Structure of fields
A field consists of FIELDNAME:FIELDCONTENTS. The colon after FIELDNAMEseems to beis mandatory.Apparently (see the previous code snippet)FIELDCONTENTSmaymust begin with a space (version/date) or a linebreak (other fields).
FIELDNAMEapparentlycan contain spaces. It should start with a capital letter -- that may not be required by the game parser, but it helps human parsers (i.e., the users of your mod).
A properly indented list structuremay not beis mandatory, but helps your users to parse the file -- especially, if you have more than just one field in your changelog entry:
Update (2019-03-12): Steinio hinted before that , possibly, no tabs may be used in changelog files. This has been confirmed now by Muppet9010. So, please, don't use tabs in the changelog file, only spaces are allowed!Code: Select all
--------------------------------------------------------------------------------------------------- Version: 0.17.0 Date: 2019.03.01 Features: - Updated to 0.17. --------------------------------------------------------------------------------------------------- Version: 0.16.0 Date: 2019.01.31 Features: - Replaced the vanilla lamp graphics for higher contrast circuit display. - Lamps are not colliding with vehicles, allowing walk or drive over. - Options on lamp base style, to fit concrete tile or for stronger contrast in signal display. - Options on lamp glow size. (Default: 3. Vanilla: 6.) - Options on reorder priority on signal colour. (Default: vanilla settings.) - Options on enable colour display on white signal. (Default: off.) - Options on enable colour display on grey signal. (Default: off.) - Options on enable colour display on black signal. (Default: off.) - Options on priority on white signal. (Default: L1.) - Options on priority on grey signal. (Default: L2.) - Options on priority on black signal. (Default: L3.) Locale: - English, German, traditional Chinese, simplified Chinese are available. License: - Using Bob's Copyright License. Document included.
Update (2019-03-19): According to my tests, the game's parser is rather finicky regarding indentation. It adheres to these rules:- "Version:" and "Date:" must start at column 1.
- All other field names must be indented by 2 spaces and start at column 3. The line must end with the colon, no space characters are allowed after it!
- Start a list (one or more lines) after the line with the field name. Each line of this list must be indented by 4 spaces and start with a dash followed by a space ("- ") at column 5. Consequently, the actual text starts at column 7. If you have very long lines, they will be wrapped so that the new lines start at column 5. You may insert a linebreak instead. In this case, you must insert at least 4 spaces to avoid parsing errors; however, you should insert 6 spaces, so the text aligns correctly.
- Notes on field names
- Be consistent! If you want to use a field name like "Bugfixes", use it always in your file -- don't write "Bugfixes: …" in one changelog entry and "Bugfix: …" in another!
- Think of these field names as categories! Seems like some cool stuff is possible if you do it right. Have a look at this screenshot, for example:
Notice the long row of tabs at the top of the screen? These correspond to field names used in the changelog file.
This screenshot just shows a list of minor features. Apparently, the parser compiles a list of all changelog entries containing a category field name ("Minor Features" in this case), and then shows the contents of that field in the tab's window. That's some quite impressive magic, isn't it? Just structure your changelog file/entries correctly and you've got filters automated -- if that isn't in the spirit of Factorio, I don't know what is …
The filtering goes even further. There is a version selector at the top left of the screen. See this screenshot:
But something is missing, right? Remember, we had the following fields defined:So, where are the tabs for "Locale" and "License"? Well, they are created -- if you set the filter to version 0.16:Code: Select all
--------------------------------------------------------------------------------------------------- Version: 0.17.0 Date: 2019.03.01 Features: - Updated to 0.17. --------------------------------------------------------------------------------------------------- Version: 0.16.0 Date: 2019.01.31 Features: - Replaced the vanilla lamp graphics for higher contrast circuit display. […] Locale: - English, German, traditional Chinese, simplified Chinese are available. License: - Using Bob's Copyright License. Document included.
Update (2019-03-08): Bilka was so kind to provide a list of FIELDNAMEs that are guaranteed to work. He also mentioned that he is not sure about custom FIELDNAMEs used by mods -- they could work, or they might cause a parsing error.
Update (2019-04-16): It seems you can really use any string you want as a field name. Bilka explained the differences between using official and custom field names: If only official field names (the ones used by the developers in the vanilla game) are used, the All tab will always be the last one on the right; if you make up your own, they will be placed to the right of the All tab, so All is harder to find.
- Empty lines
You may use empty lines to visually separate blocks. However, please make sure these lines are really empty, as in "empty string"! Neither space nor tabulator characters are allowed. Just hit ENTER to insert an empty line -- and if your editor provides an option like that, activate "Remove trailing spaces on save"!
What should go into the changelog?
So far, I've explained what is needed so your changelog will be displayed in the game. The question of what information should be mentioned there is hardly less important, however. Here are my thoughts on that matter, with suggestions for what categories to use:- Info: I'd use this for anything important that should stand out from the mass of regular changes:
- Initial release
- Update to a new major version (e.g. 0.17) of Factorio
- New or changed dependencies
Most people will have the latest version of the game installed. Some, however, might prefer to wait a bit longer to avoid newly introduced bugs -- and others may even be unable to update because of some obscure, unfixed bug. So, if you change your mod in such a way that it depends on the latest version of the base mod or a library, there may be people it might not work for after the update. If you add the changed dependencies to your changelog, these people will know that updating the mod might be dangerous before they try. If nothing else, including such information in the changelog is being polite to your users. - Bugfixes: As a rule of thumb, if a line contains the words "fixes" or "fixed", it should be filed under this category. You probably should use "Bugfixes:" even if you've only changed one bug: "Bugfixes:" is an official category, the singular form ("Bugfix:") is not -- and the in-game changelog viewer will sort tabs differently if custom categories are used (see above).
- Feature/Minor Feature/Major Feature: If you've added something you're really proud of, use these categories to make the new feature stand out!
- Changes: Almost everything else should go here, except for
- License/Translation/Locale: Again, these categories are important enough to stand out. "License:" is not an official category, however, so you should be aware that using it will mess up the order of the tabs.
General hints and common traps
- Be sure to structure all of your changelog entries correctly! A file like this:
would not throw a parsing error because it starts with a header line. However, it would be parsed differently than expected: The parser would only see "Version: 0.17.0". "Version: 0.16.0" and everything that follows would be considered as part of the description for version 0.17.0.
Code: Select all
--------------------------------------------------------------------------------------------------- Version: 0.17.0 Date: 2019.03.01 Features: - Updated to 0.17. Version: 0.16.0 Date: 2019.01.31 Features: - Replaced the vanilla lamp graphics for higher contrast circuit display. - Lamps are not colliding with vehicles, allowing walk or drive over. […]
- Test and test again, and again! If the parser aborts with an error in line 1, it doesn't mean that there are no errors later on. The parser will abort at the first error it finds, so if you correct that error and try again, it may abort at another line. Please, do yourself and your users a favor and correct everything until the file is parsed correctly!
- How to check for errors: This assumes that your mod does have a changelog file! If there is no file named "changelog.txt" in your mod's root directory, the parser won't need to be started for your mod, so there will be no debugging output. In this case, please create a changelog -- it's useful to have!
- Start the game from the command line, so you can see debugging output.
- In the main menu, click on "Mods" (or whatever that is translated to in your game's locale).
- Look up your mod under the "Manage" tab and select it.
- If your changelog file failed to parse, you will see now an error message in the debugging output. In this case, quit the game, fix the error, and restart at step 1!
Update (2019-03-21): Actually, debugging output is appended to factorio-current.log. So, you don't have to start the game from the command line if you watch that file.
- Update (2019-03-12): Don't use tabs in changelog files, indent lists with spaces instead!
- Update (2019-03-12): Again: Make sure your changelog file starts with a header line (a line that contains only dashes)!
- Update (2019-03-12): Again: The last non-empty line of your changelog file must never, ever be a line that contains only dashes!
- Update (2019-03-12): It is a lot of work to replace all those tabs with spaces, and removing all those spaces at the end of a line. Fortunately, a lot of IDEs and editors have options like "Replace tabs with spaces" or "Remove trailing spaces on save" -- use these, if you don't already! Alternatively, if you are on Linux, this oneliner will help you:
The switch "-i" causes sed to overwrite your file, so be sure to make a back-up!
Code: Select all
sed -i -e 's/\t/ /g' -e 's/\s*$//' changelog.txt
- Update (2019-04-16): Beware! "The changelog is displayed on the mod portal" is not a valid test! Apparently, if you upload your mod, the changelog file will be extracted and displayed as is -- without any regard for the syntax rules imposed by the in-game parser. So, it is possible that the changelog will fail to parse even though it is displayed on the mod portal. Use the in-game changelog viewer for testing, and check factorio-current.log for error messages, please!
Conclusion
That was all information I could think of. I hope you don't think me presumptuous for telling you how to do such easy and/or boring stuff like writing a changelog! Well, just see it this way: The devs and some modders (I don't think anybody would use every mod that is out there!) have provided me with a wonderful game that I've spent way too many hours on already, and I just want to do something for it in return -- and if this should help to improve the overall quality of the game, it's just so much the better for everyone!As stated above, there may be things I've got wrong or that are missing. If so, you are very welcome to leave your comments in the thread!
============
Changes:
2019-03-08
- Fixed some typos
- Edited section "Add a date field"
- Added link to list of guaranteed-to-work field names (section "Notes on field names")
- Added information about name/location of the changelog file (section "Structure of a changelog file")
2019-03-12
- Made my own changelog cleaner by adding dates to update notifications
- Added note about file encoding (section "Structure of a changelog file")
- Added note that tabs are not allowed in changelog files (sections "Structure of fields"/"General hints and common traps")
- For emphasis, added another note on the issue of dashed lines at start/end of changelog files (section "General hints and common traps")
2019-03-19
- Verified/corrected information I wasn't sure about before (sections "Structure of a changelog entry" and "Structure of fields")
- Added note on rules for indentation (section "Structure of fields")
- Added note on editors (section "General hints and common traps")
2019-03-21
- Added note about debugging output appended to factorio-current.log (section "General hints and common traps")
2019-03-23
- Added note about correct file encoding for changelog files (section "Structure of a changelog file")
2019-03-29
- Added note about number of dashes in header lines (section "Structure of a changelog entry")
2019-04-06
- Added section "Empty lines"
2019-04-16
- Added example of a working changelog file and a short description of the syntax rules at the top (section "In short: rules and example of a working changelog file")
- Added note on the contents of VERSION fields, regarding multiple version numbers/text add-ons (section "Structure of a changelog file")
- Added note on the difference between official and custom field names
- Added section "What should go into the changelog?"
- Added note that the mod portal is not suitable for testing changelog files (section "General hints and common traps")
2019-06-15
- Added explanation why sometimes valid changelog files would be shown in the game, but not on the mod portal (section "In short: rules and example of a working changelog file")
2019-09-21
- Added note that changelogs may now be files encoded as UTF8 with BOM (sections "In short: rules and example of a working changelog file", "Structure of a changelog file")
- Added note that "Date: " lines without a date can be made error-proof by appending at least one question mark (section "Structure of a changelog entry")
2019-12-30
- Rewrote section on Version numbers to include results of recent tests.