Merge: lib/core: provide default codec-aware read_char
authorJean Privat <jean@pryen.org>
Thu, 10 May 2018 22:46:06 +0000 (18:46 -0400)
committerJean Privat <jean@pryen.org>
Thu, 10 May 2018 22:46:06 +0000 (18:46 -0400)
commitf54946a02882c88d74808c427f851b66d796b0f1
tree2e9306a3eec260125963b7f5bac4febfd8acdb93
parent588a41fcc7c0bbcedbcd87a454ccef9a6b0d9bd4
parent6f706a41ccad5b0ba930eff7f08957f0b77ee215
Merge: lib/core: provide default codec-aware read_char

Previous implementations of read_char were unaware of codec issues, and
used to read a byte and convert it to a code point.

For ASCII characters this was enough, but once unicode characters were
read on a char-by-char basis, wrong characters would appear.

This commit fixes this issue by using the Codec API to read a character
intelligently, and properly support multibyte encodings.

Signed-off-by: Lucas Bajolet <lucas.bajolet@gmail.com>

Pull-Request: #2648
Reviewed-by: Jean Privat <jean@pryen.org>
src/interpreter/naive_interpreter.nit