<chapter id="maintainers"><title>For maintainers: maintaining internationalized &d-i; packages</title>

<sect1>
<title>
Checkout of the whole &d-i; source tree
</title>

<para>
Developers need the whole source tree, which requires the use of
<command>git</command> and <command>mr</command>:
</para>

<informalexample>
  <screen>
git clone git@salsa.debian.org:installer-team/d-i.git
cd d-i
./scripts/git-setup
mr checkout
  </screen>
</informalexample>

<para>
In the past, we had everything in one subversion repository. Then one day, the
developer parts were migrated to git, while the translator files were left
on subversion; this is why we had the project divided into two parts now.
Then, the Alioth server was replaced by Salsa, and thus everything was migrated
to git.
</para>

</sect1>

<sect1>
<title>
&d-i; packages internationalization and localization
</title>

<para>
The &d-i; packages use a <classname>debconf</classname>-managed
interface to handle user interaction screens and all display.
</para>

<para>
The debconfized packages use <classname>po-debconf</classname> for
handling localization which means that templates translations are
hosted in directories named <filename>debian/po</filename> in each
package source code tree.
</para>

<para>
These directories contain <filename>&lt;language&gt;.po</filename>
files where &lt;language&gt; is the code of supported languages. See
<xref linkend="language-codes"/> for details about language codes.
</para>

<para>
They also contain a file named <filename>templates.pot</filename>
which contains up-to-date strings. This file is generated by the
<command>debconf-updatepo</command> utility when debconf templates are
modified or written.
</para>

<para>
When working on "core" &d-i; packages, developers should not care
about the <filename>debian/po</filename> directory contents. They even
don't need running the <command>debconf-updatepo</command> utility,
except the first time they create the package (so that the
<filename>debian/po/templates.pot</filename> file is created).
</para>

<para>
The only files that are not automatically generated in <filename>debian/po</filename> are:
</para>

<itemizedlist>
  <listitem>
    <para>
      <filename>debian/po/output</filename> defines the type and
      encoding of the generated <filename>templates</filename>
      file. The file must always contain:
    </para>
    <informalexample>
      <screen>
2   utf8
      </screen>
    </informalexample>
  </listitem>
  <listitem>
    <para>
      <filename>debian/po/POTFILES.in</filename> defines the file
      containing the source strings, with a path relative to the
      <filename>debian</filename> directory of the source tree. This
      is usually:
    </para>
    <informalexample>
      <screen>
[type: gettext/rfc822deb] &lt;package&gt;.templates
      </screen>
    </informalexample>
    <para>
      where &lt;package&gt; is the name of the binary package using
      these templates.
    </para>
  </listitem>
</itemizedlist>

</sect1>
<sect1>
<title>
English templates handling
</title>

<sect2>
<title>
General recommendations
</title>

<para>
&d-i; developers are strongly encouraged to read the
<emphasis>po-debconf(7)</emphasis> man page and must have a good
knowledge of <classname>po-debconf</classname> mechanisms. Please ask
for advice on the &email-debian-boot-list; mailing list when in doubt.
</para>

<para>
Templates files <emphasis>must</emphasis> be named
<filename>&lt;package&gt;.templates</filename>. Developer
<emphasis>should not</emphasis> use files named
<filename>debian/templates</filename>, even though
<command>debhelper</command> scripts properly handle such files. This
requirement has been added because the translators work on a general
file which groups together all templates from all &d-i; packages. This
general file mentions, for each string, the originating file name. If
this file name contains the package name, the translator will then
know which package the string (s)he translates belongs to.
</para>

</sect2>

<sect2>
<title>
New debconf templates introduction
</title>

<para>
Maintainers must respect the <emphasis>Debconf Templates Style Guide
(DTSG)</emphasis> (see the section <ulink
url="https://www.debian.org/doc/manuals/developers-reference/best-pkging-practices.en.html#general-recommendations-for-authors-and-translators">General
recommendations for authors and translators</ulink> of the
Debian Developers Reference). In any case,
<emphasis>before</emphasis> committing new templates or templates
changes, these templates must have been reviewed by other &d-i;
developers or an internationalisation (i18n) coordinator.
</para>

<para>
Committing templates with incorrect English or templates with a
different writing style than the general &d-i; writing style could
trigger nearly immediate translation work. These translations will
later need to be adapted again when the strings are corrected.
</para>

<para>
For that reason, new templates for an existing &d-i; package should
initially be marked as
<emphasis>non translatable</emphasis>. This is done by
<emphasis>not</emphasis> prepending the templates fields with an
underscore character. Example:
</para>

<informalexample>
  <screen>
Template: debian-installer/foo/bar
Type: note
Description: New note.
 This is a completely new note, added for package foo.
  </screen>
</informalexample>

<para>
Packages which are newly added to &d-i; can ignore this. Their
templates may be marked as <emphasis>translatable</emphasis>, because
these packages are ignored by the synchronisation script until they
are added to the <filename>packages/po/packages_list</filename> file
(see <xref linkend="synchronisation-scripts"/>).
</para>

<para>
After adding the new templates or the new package, maintainers must
request for peer review on &email-debian-boot-list;.
</para>

<para>
New packages will be added to the master files only when this review
has been completed.
</para>

</sect2>

<sect2>
<title>
Marking strings for translation
</title>

<para>
Strings which need translation must be prepended by a leading
underscore (_) character in the <filename>templates</filename> file
after the strings review:
</para>

<informalexample>
  <screen>
Template: debian-installer/foo/bar
Type: note
_Description: New note
 This is a completely new note, added for package foo.
  </screen>
</informalexample>

<para>
Then, the synchronisation script will automatically add the two new
strings to the translators files, allowing them to work on
translations.
</para>

</sect2>

</sect1>

<sect1>
<title>
Translators work method
</title>

<para>
This paragraph describes the <emphasis>master file</emphasis>
process. It is included in the chapter aimed at package maintainers,
because it appeared important that &d-i; package maintainers
understand the way translations are handled.
</para>

<para>
All PO files from packages listed in
<filename>packages/po/packages_list</filename> are merged into five po files
per language and translators just work on these files. These
so-called <emphasis>master</emphasis> PO files are kept in
<filename>packages/po/sublevelX</filename> in the &d-i; main directory.
</para>

<para>
These files are <emphasis>the only files</emphasis> edited by
translators. All other PO files as well as PO "templates" files (POT
files) are automatically generated by the synchronization script (see
<xref linkend="synchronisation-scripts"/>):
</para>

<itemizedlist>
  <listitem>
    <para>
      merge all <filename>templates.pot</filename> files into
      <filename>packages/po/template.pot</filename> (included are all
      <filename>debian/po/templates.pot</filename> files for
      packages listed in <filename>packages/po/packages_list</filename>);
    </para>
  </listitem>
  <listitem>
    <para>
      update <filename>packages/po/sublevelX/*.po</filename> files according to
      this <filename>template.pot</filename> file;
    </para>
  </listitem>
  <listitem>
    <para>
      update all individual packages
      <filename>debian/po/*.po</filename> files with the translations
      from these single <filename>packages/po/sublevelX/*.po</filename> files.
    </para>
  </listitem>
</itemizedlist>

<para>
As a consequence of this process, changes made directly to
<filename>debian/po/*.po</filename> files are
<emphasis>lost</emphasis> in this process. So, if for some reason,
maintainers need to update these files (there is normally
<emphasis>no</emphasis> reason for this), they should consider
modifying the source files in <filename>packages/po/sublevelX/</filename>.
</para>

<para>
It is however highly recommended that such changes are discussed
with &i18n-coords; (or handed over to them).
</para>

</sect1>
<sect1>
<title>
Handling modifications to templates
</title>

<sect2>
<title>Regular modifications</title>

<para>
When maintainers modify the strings in the templates files, the
translations will generally need updates. The only exception are typo
or spelling corrections, which will be here called "trivial"
modifications.
</para>

<para>
Trivial modifications can be handled with a special process to avoid
updates to the translations. This is detailed in the next section.
</para>

<para>
For regular modifications, maintainers do not need to run the
<command>debconf-updatepo</command> utility which updates the
<filename>debian/po/templates.pot</filename> file as well as all
<filename>debian/po/*.po</filename> files. Such updates will be
handled by the synchronization script.
</para>

<para>
Modifications to templates will change the status of the translations
for the modified string: in all PO files, this string will be marked
as <emphasis>fuzzy</emphasis>. This means that, though the former
translation is kept, it is known to be
outdated. <emphasis>Fuzzy</emphasis> translations are of course not
used: a template which contains at least one fuzzy string will be
<emphasis>completely</emphasis> displayed in English. As a
consequence, maintainers should not worry about outdated translations,
this is the translators job.
</para>

<para>
This <emphasis>fuzzy</emphasis> string will be marked fuzzy in the
general PO files when the synchronization script will be run.
</para>

<para>
When modifications change the string too deeply, the translations are
not kept and thus the string becomes
<emphasis>untranslated</emphasis>. The old translation is moved as an
<emphasis>obsoleted</emphasis> entry and thus not completely lost.
</para>

</sect2>

<sect2>
<title>
Review changes to existing templates
</title>

<para>
Some peer review is strongly suggested when changing existing debconf templates
and the changes are significant enough to motivate a peer review (maintainers
should be aware that even minor changes might be inconsistent with the &d-i;
writing style, so in doubt, any change should be considered as "significant enough").
</para>

<para>
In such case, it is recommended to first duplicate the existing template, with
another name (for instance, the former name followed by <emphasis>_save</emphasis>).
</para>

<para>The changed template is then unmarked for translation (by removing the
leading "_" character) and a review may be requested in the mailing list.
</para>

<para>
As soon as the review is completed, the modified template should be marked for
translation again.
</para>

<para>
Before removing the old template, the l10n-sync script must be run once
to fill in the new template with the former translation, by <emphasis>fuzzy matching</emphasis>.
</para>

</sect2>


<sect2>
<title><emphasis>Trivial</emphasis> modifications</title>

<para>
<emphasis>Trivial</emphasis> modifications are all modifications which
have no impact on the translations: these will be most often spelling
errors or typos in original templates files.
</para>

<para>
If the update is made without care, correcting a typo in one template
will result in one more <emphasis>fuzzy</emphasis> string in all translations. 
</para>

<para>
In such case, a special <emphasis>unfuzzy</emphasis> process may be run on PO files.
</para>

<para>
As an example, let's assume that an original template contained the
following wrong English sentence:
</para>

<informalexample>
<screen>
These changes will be applied to all partitons.
</screen>
</informalexample>

<para>
If the package maintainer simply changes <emphasis>partitons</emphasis> to the correct
<emphasis>partitions</emphasis> word, all translations of the paragraph containing this
sentence will be fuzzied.
</para>

<para>
In this case, preventive <emphasis>unfuzzyfication</emphasis> may be done:
</para>

<informalexample>
<screen>
cd &lt;di_dir&gt;/packages/po/
git pull
mkdir NEW
for i in *.po; do
 msgcat --no-wrap $i |\
   sed -e '/^msgid/s/partitons/partitions/g' |\
   msgcat - &gt; NEW/$i &amp;&amp; mv NEW/$i $i
done
git add *
git commit -m"Preventive unfuzzy before correcting typo in &lt;mypackage&gt;" *.po
rmdir NEW
</screen>
</informalexample>

<para>
Then, the maintainer <emphasis>must immediately</emphasis> correct the
typo in the original <filename>&lt;package&gt;.templates</filename>
file.
</para>

<para>
<emphasis>This is a dangerous action</emphasis>. The replacement
string in the <command>sed</command> command above must be carefully
chosen so that it matches only the occurrence that is changed.
</para>

<para>
The <classname>po4a</classname> package maintainers have written a
special tool named <command>msguntypot</command> to handle
such <emphasis>preventive unfuzzyfication</emphasis>.
</para>

<para>
If in doubt, maintainers should contact the &i18n-coords; <emphasis>
before committing the error correction</emphasis> to the original
templates file.
</para>

</sect2>
</sect1>

<sect1>
<title>
String freezes
</title>

<para>
<emphasis>String freezes</emphasis> are special periods in the &d-i;
development process. They are more detailed in <xref
linkend="string-freezes"/>.
</para>

<para>
In short, during the release process of &d-i;, the release manager
will call for <emphasis>string freezes</emphasis>. A string freeze
theoretically forbids any change to templates files in &d-i;
packages. It is meant for translators to complete their work without
having it outdated by maintainers changes.
</para>

<para>
Of course, some urgent or critical changes may be needed even during a
string freeze. Maintainers needing to commit such changes should first
warn the &d-i; release manager and the &i18n-coords; and get their
approval before committing.
</para>

<para>
The &i18n-coords; will warn translators about the unplanned change
and will request them for a special update.
</para>

<para>
Always remember that &d-i; translation is the work of several dozens
of teams and thus each change adds more time for all of them to get
synchronized again.
</para>

</sect1>

<sect1>
<title>
Releasing packages
</title>
<para>
When releasing their packages, maintainers should use the
<command>scripts/l10n/output-l10n-changes -d .</command> command. This
script will output the changelog entries which should be added to the
package's <filename>changelog</filename> file, for translations which
got modifications since the last release of the package. This will
work properly <emphasis>only</emphasis> if the last changelog entry
includes "UNRELEASED".
</para>

<para>
After running the script, these lines must be added to the package's
<filename>changelog</filename> file and process with other release
tasks (which are out of scope for this document).
</para>

</sect1>

</chapter>
