Becoming a Multilingual SuperHero in Django
This is a modal window.
Das Video konnte nicht geladen werden, da entweder ein Server- oder Netzwerkfehler auftrat oder das Format nicht unterstützt wird.
Formale Metadaten
| Titel |
| |
| Serientitel | ||
| Anzahl der Teile | 50 | |
| Autor | ||
| Mitwirkende | ||
| Lizenz | CC-Namensnennung - Weitergabe unter gleichen Bedingungen 3.0 Unported: Sie dürfen das Werk bzw. den Inhalt zu jedem legalen und nicht-kommerziellen Zweck nutzen, verändern und in unveränderter oder veränderter Form vervielfältigen, verbreiten und öffentlich zugänglich machen, sofern Sie den Namen des Autors/Rechteinhabers in der von ihm festgelegten Weise nennen und das Werk bzw. diesen Inhalt auch in veränderter Form nur unter den Bedingungen dieser Lizenz weitergeben. | |
| Identifikatoren | 10.5446/44047 (DOI) | |
| Herausgeber | ||
| Erscheinungsjahr | ||
| Sprache |
Inhaltliche Metadaten
| Fachgebiet | ||
| Genre | ||
| Abstract |
|
DjangoCon US 201839 / 50
2
5
7
8
13
18
20
21
22
35
41
44
45
47
00:00
VideokonferenzService providerVererbungshierarchieProgrammfehlerOSAInternationalisierung <Programmierung>Formation <Mathematik>Web-ApplikationCanadian Mathematical SocietyProgrammfehlerREST <Informatik>App <Programm>SchnittmengeProgrammierspracheClientSystemverwaltungDefaultProjektive EbeneFront-End <Software>CASE <Informatik>CodeMereologieEin-AusgabeFormale SpracheHumanoider RoboterMailing-ListeTranslation <Mathematik>OSAHydrostatikEndliche ModelltheorieVererbungshierarchieTaskComputeranimation
01:46
MiddlewareFormale SpracheOrdnung <Mathematik>CASE <Informatik>MiddlewareSchnittmengeDateiformatGlobale OptimierungHydrostatikCompilerKlasse <Mathematik>Internationalisierung <Programmierung>MereologieProgrammierspracheVerzeichnisdienstZeichenketteDefaultMailing-ListeAttributierte Grammatik
02:53
ProgrammierspracheClientURLStrom <Mathematik>Cookie <Internet>E-MailCodeURLCookie <Internet>Formale SpracheE-MailProgrammierspracheMusterspracheWurzel <Mathematik>SchlüsselverwaltungMiddlewareDefaultCompilerBrowserClientStrömungsrichtungServerAmerican Physical SocietyOrdnung <Mathematik>AlgorithmusSchnittmengeFunktionale ProgrammierspracheRoutingComputeranimation
04:02
URLDateiformatZeichenketteTemplateMaßerweiterungElektronische PublikationSystemverwaltungTemplateInternationalisierung <Programmierung>URLE-MailClientExogene VariableDefaultInhalt <Mathematik>PasswortProgrammierspracheLastVorzeichen <Mathematik>CompilerHydrostatikKontextbezogenes SystemDateiformatMusterspracheFunktionale ProgrammierspracheEndliche ModelltheorieDatenfeldElektronische PublikationEin-AusgabeWeb-SeiteInklusion <Mathematik>CodeHilfesystemDreiecksfreier GraphStrömungsrichtungMereologieObjektorientierte ProgrammierspracheThreadFormale SpracheMiddlewareBitInformationAusnahmebehandlungTranslation <Mathematik>Schreib-Lese-Kopfsinc-FunktionWurzel <Mathematik>FehlermeldungKonfigurationsraumKartesische KoordinatenMultifunktionCookie <Internet>p-BlockFahne <Mathematik>GruppenoperationRoutingPlastikkarteREST <Informatik>Ultraviolett-PhotoelektronenspektroskopieWärmeübergangZeichenketteQuellcodeTeilmengeMomentenproblemComputeranimation
09:32
Elektronische PublikationFormale SpracheSystemprogrammTranslation <Mathematik>ZeichenketteKategorie <Mathematik>Funktionale ProgrammierspracheObjektorientierte ProgrammierspracheMultiplikationsoperatorCompilerCanadian Mathematical SocietyHydrostatikVorzeichen <Mathematik>Ultraviolett-PhotoelektronenspektroskopieSelbstrepräsentationEndliche ModelltheorieGruppenoperation
10:57
LeistungsbewertungElektronische PublikationTranslation <Mathematik>ZeichenketteFormale SpracheVerzeichnisdienstServerGasströmungSystemverwaltungKugelkappeVerzeichnisdienstMultiplikationVirtuelle RealitätSchnittmengeCompilerZeichenketteServerInternationalisierung <Programmierung>Message-PassingFehlermeldungElektronische PublikationDienst <Informatik>GeradeDateiformatProgrammierspracheDatenverwaltungFormale SpracheEin-AusgabeProjektive EbeneTranslation <Mathematik>Virtuelle MaschineObjektorientierte ProgrammierspracheWurzel <Mathematik>BildschirmmaskeDatenfeldApp <Programm>LeistungsbewertungEndliche ModelltheorieRoutingDefaultCoxeter-GruppeCASE <Informatik>DatenmodellFahne <Mathematik>CodeVersionsverwaltungWeb-SeiteInformationAttributierte GrammatikMAPBenutzerbeteiligungHook <Programmierung>MultiplikationsoperatorFortsetzung <Mathematik>HydrostatikMereologieMobiles EndgerätDatenbankTemplateKartesische KoordinatenMathematikDynamisches SystemSichtenkonzeptSchreib-Lese-KopfMomentenproblemRechter WinkelComputeranimation
18:33
TaskKontextbezogenes SystemTranslation <Mathematik>Translation <Mathematik>Formale SpracheCodeVersionsverwaltungApp <Programm>TemplateClientURLDienst <Informatik>TaskInformationFunktionale ProgrammierspracheDefaultEnergiedichteInternationalisierung <Programmierung>Kontextbezogenes SystemExogene VariableDreiecksfreier GraphCASE <Informatik>Rechter WinkelParametersystemMusterspracheRegistrierung <Bildverarbeitung>E-MailFunktion <Mathematik>
20:18
TemplateProgrammierspracheLokales NetzGewöhnliche DifferentialgleichungWeb-SeiteProgrammierspracheFormale Sprachep-BlockWeb-SeiteCodeMessage-PassingTemplateComputeranimation
21:02
TemplateProgrammierspracheWeb SiteFormale SpracheVerschlingungComputeranimation
21:21
Internationalisierung <Programmierung>Formale SpracheURLTemplateProgrammierspracheCodeHilfesystemVerschlingungGarbentheorie
22:01
Nabel <Mathematik>Fuzzy-LogikZeichenketteMAPTranslation <Mathematik>ZeichenketteCodeZahlenbereichInternationalisierung <Programmierung>Formale SpracheCompilerApp <Programm>ProgrammierspracheElektronische PublikationJensen-MaßOpen SourceSelbstrepräsentationMessage-PassingMomentenproblemInstantiierungVersionsverwaltungHyperbelverfahrenNabel <Mathematik>DifferenteMultiplikationPunktProjektive EbeneTwitter <Softwareplattform>HydrostatikMultiplikationsoperatorFunktion <Mathematik>Gemeinsamer SpeicherGeradeComputeranimation
24:55
COMService providerDatentypJSONXMLComputeranimation
Transkript: Englisch(automatisch erzeugt)
00:00
Thank you so much. Let me just introduce myself first.
00:24
I'm one of you, a part of the community. I am a CPython contributor. I am Bhaktriyaj access on bugs.python.org. I've been a contributor to Mozilla's ecosystem of projects ranging from add-ons, marinate, task cluster, gecko engine, et cetera.
00:40
I've been a GSOQ mentor for Debian and now GSOQ mentor. I work as a backend dev at Fuel, empowering iOS and Android and web apps. This talk came directly from one of the projects I worked on which was for a Chinese client. So imagine you've got this awesome RESTful API
01:01
built on Django and DRF and suddenly this requirement comes in. We want this app and the CMS or the admin panel in Chinese or we want it in German. So if you have this humongous code base, and it's not written with all those you get text and you get text lazy stuff in mind, then boy, you're in danger. And you should better start praying to the almighty
01:20
to give you the strength and endurance on your part to become a modeling or super hero. The first and foremost thing to enable translation is to tell Django what is the list of languages that you support, where does it find the translation for static data, and what is the default language it should fall back on
01:41
in case there is no translation available for the requested language. So we'll have a look at the settings. So we include a bunch of settings. First is the middleware classes. We have the special middleware which is local middleware which is placed just after the session middleware and before the common middleware. The order of the middleware is quite important
02:03
and we'll see next how it plays a role. We define the list of languages. So in this particular example, I am supporting English, simplified Chinese, and traditional Chinese. We also defined some more attributes. One of them is use i18n,
02:23
which essentially makes some optimization to load all the internationalization machinery that helps you in localization. We make use l10n to true, which will help you format dates, numbers, calendars according to the current locale. We set the default language to English in this case
02:42
and then we define the locale parts. So this is our directory where all your static string translations goes. So we'll see next how it plays a role. But first we need to decide how to track the language the client wants to communicate in.
03:01
So there are different ways to get the preference in an HTTP request and in order to retrieve the language preference from client's request, local middleware tries to determine the user's language preference by the following algorithm. So first it would look for language prefix in the requested URL. So for example, if your URL is slash EN slash APS
03:21
slash resource, EN stands for the language code. And this is only performed when you're using i18n patterns function in your root URL conf. Failing that, it looks for the language session key in the current user session. Failing that, it looks for a cookie. The name of the cookie used is set by the language cookie name setting.
03:44
The default name is Django language. Failing that, it looks for the except language HTTP header. This header is sent by your browser and tells the server which languages you prefer in order of priority. Django will try each language in the header until it finds one with the available translations.
04:03
For the REST APIs, I personally have found the except language header a much cleaner way to accomplish the task. And for enabling multiple languages for the admin panel routes, since it's a get request, we'll prefer having i18n patterns function
04:20
and modify the root URL conf as shown here. So all we say is, we describe the URL here and we just wrap it up in the i18n patterns function. So once we do this, the i18n patterns will automatically prepend the current active language code to all the URL patterns defined
04:41
within the i18n patterns function. So all your admin URLs with the current configuration having zitcn and enactivated will have the URLs as shown here. So the start here indicates that anything can come over as a suffix. So for whatever language you want the end panel to be accessible, the use of corresponding language code
05:02
in the URL can help you in accessing that. There is also a flag which is known as prefix default language. Once it is set to false, the default language code will not be prepended. And you can access the admin route at just slash admin.
05:20
Although you can do this to all the URLs, but for the API endpoints, we prefer to supply this bit of information in except language HTTP header. So gotchas. Throughout this talk, I'll go through some of the gotcha moments, which I personally felt a bit annoying,
05:40
like I was completely banging my head. So we'll see. The locale middleware should always come before the common middleware and after the session middleware. And why is that? Because as we see in the HTTP request, how do you pass HTTP request, then the session middleware would first set the language code in the cookie
06:01
if you're using that particular method, and then only the locale middleware can, the locale middleware could actually process that information and take the language code from there. So what does the locale middleware help with? It's a secret sauce in the translation machinery. Let's have a look at the Django's request response cycle
06:23
to understand it in a better way. It does the following for the request part. It passes it and decides what translation object to install in the current thread context. For the response part, it does two things. So first of all, it will set the content language header
06:41
in the response for the client to know what language is used in the response. So for example, if a client is requesting German language but we do not support German in this example, then the default fallback will be on English, and in the content language header we say that this particular response is in English.
07:00
And the second thing it does is, it formats the URL with activated language if the i18n patterns function is used. So it will say slash en slash whatever. So the question is what to translate? Majorly, there are two kinds of data. So one is the static data, which will include all the model names, field name of models, error messages
07:23
that will be starting in the application. When we come to dynamic data, it essentially includes the field value in models that will be input by the user. So for static strings, we would need model names, field names, error messages, and we use them in many places,
07:42
in Django templates, all of our code files, the models.py, and let's see how we support those translation in all those places. So anything that should be translated in templates should be marked with either trans or trans block tag. To make the translation work,
08:00
there is one more thing to take care of, and that is loading the internationalization i18n tag at the top of every file that uses trans or block trans. So let's see it in action. I have a dummy page which says sign up. It includes a username and password label, and have input for username and password.
08:22
Pretty simple. Now, how do I make internationalization and localization here? So I include, to enable the translation and serve the static page, I'll first load the translation tag and mark all the stuff with translation.
08:41
So I've included the i18n tags. Now, for all the static strings, I mark them with the trans tag. So I say sign up, username and password are marked for translation. If there are translation available, then please go ahead and replace these with those static translations. Gotcha, too.
09:00
The i18n tag should be loaded in every file even if it extends other file that already has it loaded. Now, this particular thing can leave you bizarre. Always remember to include i18 tag at top of all the templates that uses translation. If you don't do it, you'll keep pulling out your hairs
09:21
and wondering what happened, but it won't work. So let's discuss about how to support translations in all .py files. So we'll talk about models. So I have this user model here which has the verbose name defined with the ugettext lazy function.
09:41
So if you see the import at the top of the file, it's from django.utils or translation import ugettextlazy at underscore, and all the verbose name property are marked with ugettextlazy. So wherever they occur in your CMS or whatever, the translations will be picked up.
10:01
So both ugettext and ugettextlazy are just Python objects that are evaluated to string at different times. So the string representation depends on whichever language is being activated. Let's see how, let's see it in action. So I imported ugettext,
10:21
and then I imported a bunch of utility functions, activate and get language here. So first of all, I do activate en. So I activated English language just to make sure that it worked. I used the getlanguage function to know that, okay, English is activated. Then I called ugettext for signup, and since the translation was there,
10:42
I get the English representation. Similarly, I did it for simplified Chinese. So I activated it, I tried to get language, and then I tried to get the translation. We'll see where do we actually put those statics and translations later. So the idea is like,
11:02
how do we decide which one to use? Should we use ugettext or should we use ugettextlazy? Well, if you need immediate evaluation of the translation, then always use ugettext. For example, in all your views.py file, because once the request is coming in, you need immediate evaluation, so use ugettext.
11:23
And use ugettextlazy for lazy referencing the string object, for example, in your models.py. So now that we have all the static data marked with trans tag, ugettext, and ugettextlazy methods,
11:41
it's time to generate the translation files and fill in the translations. So all the translations goes in Django.po file. PO stands for portable object format. And for generating the Django.po file for simplified Chinese, we'll run the following management command. We'll say python manage.py make messages.
12:01
Now we give a flag, which is minus L, indicating the language code. So once we do this, we'll see something like this. In our project directory, there would be a folder named as locale. So if you remember in the settings, we already mentioned that what is the path of the locale folder.
12:21
So this is how it comes into play. So we have locale folder. Inside that, we have this zhcn folder defined, because we just executed the command. Inside it, we get locale messages directory, and inside it, we get the Django.po file. Done.
12:41
Now, if we have a look at the Django.po file, considering the static template that was shown earlier, the signup page, we'll have something like this. So this is automatically generated by Django. It has message ID and message STR. So message ID is the text marked for translation,
13:03
and the message STR is its translated form. You have to do this for all the languages that you want to support, and for each respective language, there would be a different Django.po file. So let's fill in this for the Chinese language. So once I fill this in, I'll get something like this.
13:24
And so far, so good. Gotcha, four. While running the make messages command, if you have the virtual environment placed in the root Django directory,
13:43
then it would kind of try to generate Django.po files for all the packages that are in virtual environment. So don't make this mistake. Place your virtual environment out of your Django project. And this might not seem as a big problem right now, but we'll see how it can be potentially very big problem.
14:03
So now, this is the crux of the whole presentation, which took me like three days to figure it out. I was banging my head like, I kept on reading the documentation, and I was like, okay, let's see, let's see if it's not working, why is it so?
14:20
So notice that when you mention the language in settings, you do it by language name. So in this case, we have zh, which stands for Chinese, and cn for simplified Chinese, and this is all in lowercase separated by hyphen. But when you did this in make messages command, you used locale name, which was zh underscore cn.
14:43
So cn is the region, and it should be in caps, separated by underscore. If you make a mistake here, you won't see any error, and your translations won't work either. There is a small caveat, there's a small line in the documentation which says,
15:01
this is how you should do it, so take care here, and yeah. So we have defined all the static string translations, and now we'll see how to compile these static string translations. We'll run the command python manage.py compile messages, and as soon as we do that,
15:21
this will generate Django.mo files corresponding to all the Django.po files that are there. Django, the mo file stands for machine readable object. As soon as this is generated, Django can quickly pick it up and show the static string translations. Gotcha six, the compile messages command
15:41
would generate the Django.mo files for every package in a virtual moment if it is in the root directory of your project. So I made this mistake, I had the virtual moment inside my root directory, and then once I ran the compile messages command, it was failing, and I was like, why is it failing?
16:01
So I threw the trace back, I realized that okay, this was particularly some other external package, and it was trying to generate the Django.mo files for the packages that are in my virtual moment. So gotcha seven, always restart the whiskey server
16:20
after compile messages, otherwise the translations won't work. So let's go on to the dynamic string translations. So most of the data in our Django application is dynamic and user-generated. We can employ two approaches in supporting translations. So first is like enable the end user
16:41
to enter information in multiple languages. So we have to take care of how do we make the changes in the database for this. And the second one is like translating the dynamic text using third-party services such as Transifex. So Transifex is nothing, it's a third-party service, you just dump all your data,
17:01
and there are actual translators in place which translate it and gives all the data back through Webhook which you can store it. So we'll just see the first approach because the second approach is dependent on the first one. So let's do it for the model fields. So the tricky part begins here. Now you want to support multilingual data
17:21
in your databases and let's assume that the user can input in just two languages, say Chinese and English for simplicity. And we can use the Django model translation package here which will create columns for each of the attributes that are marked for translations. If you see this code, this is very similar to what you write for Django admin
17:41
for any data model in your app. So you just say that I want to translate first name and last name in whatever languages I have defined in the settings. As soon as you do that and if you see the SQL behind it it is something like this. So here the first name is the default field that was defined on the model
18:00
and which tends to store and retrieve the value of the first name for the default language set in the Django app which is English in this case. And for each subsequent language that our Django project supports, a new field with the same name appears so fixed with the language code it is created. So it is first name underscore zh underscore cn for simplified Chinese version.
18:21
So what if you don't want to burden your user with adding information in multiple languages? You can use a third party service such as Transifex for all the incoming data as we discussed. So since you all are still here full of energy here are some bonus tips and tricks for internationalization and debugging
18:41
translation issues for you. We discussed that the request response cycle exactly knows which language is being requested by the client but what if you are using async tasks such as the celery queue? How does your async task know what language
19:00
was there when it was called? So whenever the async task is called the caller should supply the info for the language. Otherwise we'll have the default language already set. So I have this send company registration email task here where I have defined the default language code will be in Chinese because this was a Chinese app.
19:21
So in case like if a user is requesting English this should be override once the caller is calling this particular task. So this is a service function which is being called by the task and it accepts the language code. So what it does is it first of all activates the language code so that whenever the template is rendered it is rendered in that particular language.
19:45
Secondly, in the context itself it populates all the URLs that should be exactly matching the same language code that was in the request. So we use something known as translate URL here.
20:01
We give it the URL as well as the language code. So this language code right in the parameter goes here and then it knows like if it is the Chinese version of the template the URL also points to the Chinese version which is through the iodine and pattern function. So what happens when you switch language in templates?
20:23
So we have something known as get current language. We use that and we get the language code here. So assume that this page in Chinese. So we'll get the welcome to our page message in Chinese. But for the second block which is this one
20:42
we have overridden it and said that okay we have used language EN and said that okay this particular block should override the English language. So even if this particular page is in Chinese the first message will be in Chinese but the second one, the welcome to our page will be in English because we have used
21:00
the language template tag to override that. Now how do you support multiple languages in templates? So assume that you have a website and you want to display like we support these many languages and you just click on the link and it will redirect you to the particular website that supports that language.
21:22
So if you're localized URLs get reversed in templates they always use the current language. This is very important. And to link to a URL in another language we use the language template tag. It enables the given language in the enclosed template section as shown. We use the get available languages.
21:41
We get them in the languages variable. We can then iterate on it and we get the language code and language name. As shown in the previous one we just override the language code with the help of the language template tag and as soon as we do that we just keep on entering the links.
22:01
That's it. Gotcha eight. Check in the Django shell if the translations are working with activate and you get text. We already discussed it. You can just activate a language, get a you get text representation so that you know if it is working or not. Gotcha nine. So some strings are still not being translated.
22:21
If you go in your Django dot PO files they would have been marked as fuzzy. So these are particularly strings that Django marked as fuzzy because they want your translator to have a second look at it. Once you're sure that the translation is perfect then you can remove that fuzzy line
22:41
and compile the messages again and the translation would work. So conclusion. Django's translation support is indeed very powerful but the initial setup becomes a lot of pain due to simple gotcha moments which we discussed and it can potentially cause a lot of headaches and sometimes pulling out your hair.
23:01
The more early you support your project in different languages and write the code correctly the easier it would be in the future to support multiple languages. Oh, do you know something? You just became a multilingual superhero with apps supporting multiple languages. Congratulations. So I'm also writing this book
23:21
which is leveling up your Python skills. It would be a free and open source book. The alpha version should be live till May 2019. You can subscribe to updates if you want and I'll be available on these platforms. You would find the articles on multilingual translations
23:40
on my medium and here's my mail, here's my code mentor ID, GitHub and Twitter. Share love. Thank you. So there was a step that you ran to auto-generate the strings that needed to be translated
24:00
and that was for the particular locale, correct? What happens if you change some strings and you add some strings later? Does it overwrite the entire file? Does it keep the strings that are already around? So Django is particularly very smart around that concept. So say I ran the file at a particular instance
24:22
and it includes say three strings for static translation. Then at later point of time I included more strings and then whenever I run that particular command again it won't overwrite it but it would include all those things. And at each of the step in the comments
24:41
it also mentioned the line number and the file at which the particular translation occurs. So it would check that and it would overwrite that if it is needed. Thank you.