mirror of
https://github.com/tiennm99/DocsGPT.git
synced 2026-10-03 18:46:54 +00:00
81b6ee5daa
* feat: postgres tests * feat: mongo cutoff * feat: mongo cutoff * feat: adjust docs and compose files * fix: mini code mongo removals * fix: tests and k8s mongo stuff * feat: test fixes * fix: ruff * fix: vale * Potential fix for pull request finding 'CodeQL / Clear-text logging of sensitive information' Co-authored-by: Copilot Autofix powered by AI <62310815+github-advanced-security[bot]@users.noreply.github.com> * fix: mini suggestions * vale lint fix 2 * fix: codeql columns thing * fix: test mongo * fix: tests coverage * feat: better tests 4 * feat: more tests * feat: decent coverage * fix: ruff fixes * fix: remove mongo mock * feat: enhance workflow engine and API routes; add document retrieval and source handling * feat: e2e tests * fix: mcp, mongo and more * fix: mini codeql warning * fix: agent chunk view * fix: mini issues * fix: more pg fixes * feat: postgres prep on start * feat: qa tests * fix: mini improvements * fix: tests --------- Co-authored-by: Copilot Autofix powered by AI <62310815+github-advanced-security[bot]@users.noreply.github.com> Co-authored-by: Siddhant Rai <siddhant.rai.5686@gmail.com>
29 lines
1.2 KiB
Plaintext
29 lines
1.2 KiB
Plaintext
Notes on multilingual text handling.
|
||
|
||
These notes illustrate a handful of typographic and script considerations
|
||
that software dealing with international text must keep in mind. The
|
||
examples intentionally mix several writing systems so that a parser can be
|
||
exercised end to end.
|
||
|
||
Latin script covers English, French, German, Spanish and many more. The
|
||
em-dash — together with the en-dash – is used for pauses and
|
||
ranges. Curly quotes such as “hello” and ‘world’ are
|
||
preferred in polished prose, while straight quotes remain common in
|
||
source code. An ellipsis character … compresses three dots into a
|
||
single glyph.
|
||
|
||
Cyrillic script is used for Russian, Ukrainian, Bulgarian and several
|
||
other languages. A short sample: Привет,
|
||
мир. Это простой
|
||
тестовый текст.
|
||
|
||
CJK ideographs cover written Chinese, Japanese and Korean. A short
|
||
sample of each: 你好世界. こんにちは世界.
|
||
안녕하세요 세계.
|
||
|
||
Punctuation worth noting includes the middle dot ·, the section sign
|
||
§, the pilcrow ¶ and the interrobang ‽. A well-behaved text
|
||
pipeline preserves each of these exactly, normalises line endings to a
|
||
single convention, and never silently replaces characters it does not
|
||
understand.
|