[ANN] Project Fluent for Haskell

Announcements
1

We are happy to announce that Project Fluent has finally come to Haskell! :tada:

Fluent is a localisation system for natural-sounding translations. It keeps simple messages simple and lets translators express plurals, gender and other grammar when a language needs it.

Core:

  • fluent-syntax
    The FTL syntax tree and parser, as well as a quasi-quoter to construct resources at compile time. Tested against the upstream reference fixtures from the Fluent project.
  • fluent
    Translation API comprised of Bundle, Locale, and Translate. This library does not provide any concrete instances of Locale, and is thus not intended to be used directly for translating messages.
  • fluent-effectful
    A Fluent effect for the effectful effect system. Can be either single or multi language. Just like fluent, it does not contain any concrete instances of Locale.

Backends:

  • fluent-icu
    A Locale backed by text-icu. This library re-exports everything needed to translate messages.
  • miso-fluent
    In-browser localisation for miso apps, backed by the browser’s Intl API.

All five packages are version 1.0.0 and licensed under EUPL-1.2.

For usage examples, please have a look at the module documentation on Hackage.

These libraries power our production systems, so we are committed to long-term, responsible maintenance. We are very much looking forward to your feedback, feature requests, and bug reports. :slightly_smiling_face:

Why Fluent?

Most localisation tools map each translation one-to-one onto the English string, so English grammar limits every other language.

Let’s take an example from Firefox, adapted from Mozilla’s Fluent 1.0 announcement[1].

We add one line to en.ftl:

tabs-closing = You are about to close { $count } tabs.

In Czech, “tab” is panely for counts of 2-4 and panelů otherwise. The Czech translator adds the following to cz.ftl:

tabs-closing = { $count ->
    [few] Chystáte se zavřít { $count } panely.
   *[other] Chystáte se zavřít { $count } panelů.
}

The Haskell code is the same for every language:

translate "tabs-closing" ("count", 3 :: Int)
-- Right "Chystáte se zavřít 3 panely."

The developer only supplies an identifier and variables, while the translator is free to decide the complexity of the translation for the given language. Complexity (or lack thereof) in one language does not affect the code or other languages. It’s nice not having to glue together strings and conditionals in code.

Mozilla calls this asymmetric localisation. It also covers grammatical case, gender, and more.

Haskell can have nice things, too!

Mozilla’s official Rust implementation, fluent-bundle[2], resolves numbers without locale formatting[3].

On the Haskell side of things, fluent-icu and miso-fluent format numbers, currencies, units and dates for the locale:

translate "price" ("amount", 1234.5 :: Double) enGB
-- Right "1,234.50 euros"
translate "price" ("amount", 1234.5 :: Double) deCH
-- Right "1'234.50 Euro"

This makes our Haskell implementation more feature-complete than the official Rust one. :slightly_smiling_face:

Is this LLM slop?

No! Where would be the fun in that? We take great pride in our work[4]. We do use LLMs sometimes[5], but every line of code in our codebase is thoroughly reviewed, rewritten, argued about, rewritten again … by real humans.

Speaking of real humans, I’d like to take this opportunity to thank the following ones for all the reviewing and arguing and rewriting over the months, and without whom these libraries would never have seen the light of day:

  1. Fluent 1.0: a localization system for natural-sounding translations - Mozilla Hacks - the Web developer blog ↩︎

  2. crates.io: Rust Package Registry ↩︎

  3. In types/number.rs, FluentNumber::as_string is self.value.to_string() plus zero-padding for minimumFractionDigits. style, currency, and currencyDisplay are parsed and stored in FluentNumberOptions, but nothing reads them when formatting. There’s no locale-aware digit grouping and no DATETIME builtin. The same file has a // XXX: Add support for other options. comment on the plural side. ↩︎

  4. or maybe we just like pain and have too much time to spare ↩︎

  5. after all, how else can you complain about them ↩︎

32 Likes
2

How is the Value class supposed to work? It is giving me some really large strings if I plug in NaN.

ghci> let Right x = translate "temperature" ("degrees", value @Double (0/0)) german :: Either String Text
ghci> putStrLn $ unpack x
-200’000’000’000’000’000’000’000’000’000’000’000’000’000’000’000’000’000’000’000’000’000’000’000’000’000’000’000’000’000’000’000’000’000’000’000’000’000’000’000’000’000’000’000’000’000’000’000’000’000’000’000’000’000’000’000’000’000’000’000’000’000’000’000’000’000’000’000’000’000’000’000’000’000’000’000’000’000’000’000’000’000’000’000’000’000’000’000’000’000’000’000’000’000’000’000’000’000’000’000’000’000’000 Grad Celsius
3

The current implementation uses Scientific.fromFloatDigits to convert the given Double value to Scientific:

ghci> import Data.Scientific qualified as Scientific
ghci> Scientific.fromFloatDigits (0 / 0 :: Double)
-2.0e308
4

But what is the class supposed to do? It has no documentation, and the instances seem arbitrary. For example, fromFloatDigits supports any RealFloat, but Value doesn’t have an instance for every RealFloat. It reminds me of Convertible.

5

The class represents every type we know how to format, by going through SomeValue as a concrete representation of values and relevant formatting options.

It’s true that the instances are incomplete and arbitrarily chosen (e.g. why is it missing Float). They’re just the ones we thought of while implementing this. PRs welcome. :slight_smile:

I think that especially in the case of Double and Float, the Value class could earn its keep by implementing a custom format that checks for NaN.

3 Likes
6

I used Fluent many years ago for a (js) project, and really liked it. So much better than gettext or anything else I’ve tried for translations.

1 Like
7

Congratulations on this release @ners ! It looks very nice.

3 Likes
8

These are good news!

The formerly common solution hgettext had the problem that it depended on external C code and that this C code stores the locale in a global variable. It seems that Fluent is written in pure Haskell, but it seems we need additional backends that in turn may require C libraries with global variables.

I found haskell-gettext to be a cleaner alternative, though its version bounds are currently not up-to-date.

1 Like
9

Do you like to announce that package at haskell@haskell.org, too?

1 Like
10

That’s a great point, and exactly why we designed fluent with this split frontend/backend architecture. The Locale class is open to allow you to bring locale data in however you like. text-icu is one such provider, but you can also do so entirely in pure Haskell.

Here is an example from @rumaan’s MC3000 project:

We’re actually planning to add a more complete English example locale (with number rules, decimal formatting, date and time formatting, common units and currencies) in 1.1.0.

We also have a mostly complete pure Haskell implementation of CLDR, using codegen to parse Unicode CLDR data and emit Haskell structures that can then be used to implement fluent-cldr entirely in-memory. And without Template Haskell too.

The reason we haven’t published it yet is because CLDR is big, and we want to choose an appropriate way to represent all that data without exploding your dependency closure / compiled program size. Maybe we need to find a way expose ways to build just a subset of locales? If this sounds like an interesting problem to you, definitely get in touch. IDA is always looking for more contributors. :slight_smile:

2 Likes
11

I am trying to analyse syntax errors with the fluent-syntax parser, but without luck:

ghci> AttoText.parseTest Parser.entry $ Text.pack "welcome = hallo"
Partial _

I see parseOnly works better, but parse error messages are pretty useless:

ghci> AttoText.parseOnly Parser.entry $ Text.pack "welcome = { hallo"
Left "Failed reading: empty"

Maybe there is some tooling for checking Fluent files?

12

Good news everyone! :raised_back_of_hand::nerd_face::raised_back_of_hand:

We’ve released version 1.1.0 of the fluent packages, which address the excellent feedback from this thread:

ghci> format (pure English) (toValue @Double 1234.5)
Right "1,234.5"
ghci> format (pure English) (toValue @Double $ 1 / 0)
Left "Infinity"
ghci> format (pure English) (toValue @Double $ acos 2)
Left "NaN"

And as a special treat for @thielema:

  • fluent-lint 1.0.0
    A command-line tool that parses Project Fluent translation files and checks for syntax errors and consistency issues.
$ fluent-lint de.ftl

de.ftl
  parses [✘]

Failures:

  app/Lint.hs:16:42: 
  1) de.ftl parses
       de.ftl:1:18: Expected token: "}"
       1 | welcome = { hallo
         |                  ^

  To rerun use: --match "/de.ftl/parses/" --seed 1806415045

Randomized with seed 1806415045

Finished in 0.0004 seconds
1 example, 1 failure

We’ve adapted the parser to carry location information and produce better error messages.

Take it for a spin! I’m excited to hear more feedback.

7 Likes
13

Just for consistency: you could have either “Inf” + “NaN” or “Infinity” + “Not a Number”. But you have a mix of formats.

14

Infinity and NaN is what Haskell itself uses:

>>> 1 / 0
Infinity
>>> 0 / 0
NaN
1 Like
15

Would you mind adding fluent to Stackage?