A2l File Format Specification

[Magnhild Lachowicz]

Thebuilt-in string class provides the ability to do complex variablesubstitutions and value formatting via the format() method described inPEP 3101. The Formatter class in the string module allowsyou to create and customize your own string formatting behaviors using the sameimplementation as the built-in format() method.

This function does the actual work of formatting. It is exposed as aseparate function for cases where you want to pass in a predefineddictionary of arguments, rather than unpacking and repacking thedictionary as individual arguments using the *args and **kwargssyntax. vformat() does the work of breaking up the format stringinto character data and replacement fields. It calls the variousmethods described below.

Loop over the format_string and return an iterable of tuples(literal_text, field_name, format_spec, conversion). This is usedby vformat() to break the string into either literal text, orreplacement fields.

The values in the tuple conceptually represent a span of literal textfollowed by a single replacement field. If there is no literal text(which can happen if two replacement fields occur consecutively), thenliteral_text will be a zero-length string. If there is no replacementfield, then the values of field_name, format_spec and conversionwill be None.

Retrieve a given field value. The key argument will be either aninteger or a string. If it is an integer, it represents the index of thepositional argument in args; if it is a string, then it represents anamed argument in kwargs.

Implement checking for unused arguments if desired. The arguments to thisfunction is the set of all argument keys that were actually referred to inthe format string (integers for positional arguments, and strings fornamed arguments), and a reference to the args and kwargs that waspassed to vformat. The set of unused args can be calculated from theseparameters. check_unused_args() is assumed to raise an exception ifthe check fails.

The str.format() method and the Formatter class share the samesyntax for format strings (although in the case of Formatter,subclasses can define their own format string syntax). The syntax isrelated to that of formatted string literals, but it isless sophisticated and, in particular, does not support arbitrary expressions.

In less formal terms, the replacement field can start with a field_name that specifiesthe object whose value is to be formatted and insertedinto the output instead of the replacement field.The field_name is optionally followed by a conversion field, which ispreceded by an exclamation point '!', and a format_spec, which is precededby a colon ':'. These specify a non-default format for the replacement value.

The conversion field causes a type coercion before formatting. Normally, thejob of formatting a value is done by the __format__() method of the valueitself. However, in some cases it is desirable to force a type to be formattedas a string, overriding its own definition of formatting. By converting thevalue to a string before calling __format__(), the normal formatting logicis bypassed.

A format_spec field can also include nested replacement fields within it.These nested replacement fields may contain a field name, conversion flagand format specification, but deeper nesting isnot allowed. The replacement fields within theformat_spec are substituted before the format_spec string is interpreted.This allows the formatting of a value to be dynamically specified.

width is a decimal integer defining the minimum total field width,including any prefixes, separators, and other formatting characters.If not specified, then the field width will be determined by the content.

When no explicit alignment is given, preceding the width field by a zero('0') character enablessign-aware zero-padding for numeric types. This is equivalent to a fillcharacter of '0' with an alignment type of '='.

The precision is a decimal integer indicating how many digits should bedisplayed after the decimal point for presentation types'f' and 'F', or before and after the decimal point for presentationtypes 'g' or 'G'. For string presentation types the fieldindicates the maximum field size - in other words, how many characters will beused from the field content. The precision is not allowed for integerpresentation types.

In addition to the above presentation types, integers can be formattedwith the floating-point presentation types listed below (except'n' and None). When doing so, float() is used to convert theinteger to a floating-point number before formatting.

Fixed-point notation. For a given precision p,formats the number as a decimal number with exactlyp digits following the decimal point. With noprecision given, uses a precision of 6 digits afterthe decimal point for float, and uses aprecision large enough to show all coefficient digitsfor Decimal. If no digits follow thedecimal point, the decimal point is also removed unlessthe # option is used.

General format. For a given precision p >= 1,this rounds the number to p significant digits andthen formats the result in either fixed-point formator in scientific notation, depending on its magnitude.A precision of 0 is treated as equivalent to aprecision of 1.

With no precision given, uses a precision of 6significant digits for float. ForDecimal, the coefficient of the resultis formed from the coefficient digits of the value;scientific notation is used for values smaller than1e-6 in absolute value and values where the placevalue of the least significant digit is larger than 1,and fixed-point notation is used otherwise.

For float this is the same as 'g', exceptthat when fixed-point notation is used to format theresult, it always includes at least one digit past thedecimal point. The precision used is as large as neededto represent the given value faithfully.

Template strings provide simpler string substitutions as described inPEP 292. A primary use case for template strings is forinternationalization (i18n) since in that context, the simpler syntax andfunctionality makes it easier to translate than other built-in stringformatting facilities in Python. As an example of a library built on templatestrings for i18n, see theflufl.i18n package.

$identifier names a substitution placeholder matching a mapping key of"identifier". By default, "identifier" is restricted to anycase-insensitive ASCII alphanumeric string (including underscores) thatstarts with an underscore or ASCII letter. The first non-identifiercharacter after the $ character terminates this placeholderspecification.

Performs the template substitution, returning a new string. mapping isany dictionary-like object with keys that match the placeholders in thetemplate. Alternatively, you can provide keyword arguments, where thekeywords are the placeholders. When both mapping and kwds are givenand there are duplicates, the placeholders from kwds take precedence.

Like substitute(), except that if placeholders are missing frommapping and kwds, instead of raising a KeyError exception, theoriginal placeholder will appear in the resulting string intact. Also,unlike with substitute(), any other appearances of the $ willsimply return $ instead of raising ValueError.

Advanced usage: you can derive subclasses of Template to customizethe placeholder syntax, delimiter character, or the entire regular expressionused to parse template strings. To do this, you can override these classattributes:

Alternatively, you can provide the entire regular expression pattern byoverriding the class attribute pattern. If you do this, the value must be aregular expression object with four named capturing groups. The capturinggroups correspond to the rules given above, along with the invalid placeholderrule:

Split the argument into words using str.split(), capitalize each wordusing str.capitalize(), and join the capitalized words usingstr.join(). If the optional second argument sep is absentor None, runs of whitespace characters are replaced by a single spaceand leading and trailing whitespace are removed, otherwise sep is used tosplit and join the words.

For a project, I need to work with varying types of files from some old games and related software--configuration files, saves, resource archives, and so on. The bulk of these aren't yet documented, nor do tools exist to work with them, so I must reverse-engineer the formats and build my own libraries to handle them.

Although I don't suppose there's great demand for most of it, I intend to publish the results of my efforts. Are there any accepted standards for documenting file formats? Looking around, there are several styles in use: some, like the .ZIP File Format Specification, are very wordy; others, like those on XentaxWiki, are much more terse--I find some of them difficult to read; the one I personally like best is this description of the PlayStation 2 Memory Card File System, which includes both detailed descriptive text and several 'memory maps' with offsets and such--it also most closely matches my use case. It will vary a little for different formats, but it seems there should be some general principles that I should try to follow.

I may have some old piece of software which stores its configuration in a 'binary' file--a series of bitfields, integers, strings, and whatnot all glued together and understood by the program, but not human-readable. I decipher this. I wish to document exactly what is the format of this file, in a human-readable way, as a specification for implementing a library to parse and modify this file. Additionally, I'd like this to be easily understood by other people.

There are several ways such a document might be written. The PKZIP example above is very wordy and mostly describes the file format in free text. The PS2 example gives tables of value types, offsets, and sizes, with extensive comments on what they all mean. Many others, like those on XentaxWiki, only list the variable types and sizes, with little or no commentary.

I ask whether there is any standard, akin to a coding style guide, which provides guidance on how to write this kind of documentation. If not, is there any well-known excellent example that I should emulate? If not, can anyone at least summarize some useful advice?

3a8082e126