v0.8.0-rc.1

Hi,

大家好，從 2022 年 4/29 開始的 Rust Rewrite (89ef2a8) 終於進入一個新的里程碑！
0.8.0 將會是第一個完全用 Rust 重寫的 libchewing 的第一個 release，希望大家幫忙
多多測試。

重大變更：新字典格式與舊有工具不相容

現有使用者字典將會自動備份及轉移，但舊有工具將無法直接操作字典檔案。
請使用新的 chewing-cli 工具來檢視、傾出或建構使用者字典。最終版本將提供更新的文件說明。

Download the source tarball from github releases:

https://github.com/chewing/libchewing/releases/tag/v0.8.0-rc.1

libchewing releases can be verified with the following minisign public key

minisign -Vm libchewing-0.8.0-rc.1.tar.zst \
         -P RWRzJFnXiLZleAyCIv1talBjyRewelcy9gzYQq9pd3SKSFBPoy57sf5s

What's New in libchewing 0.8.0-rc.1 (UNRELEASED)

BREAKING CHANGE: New dictionary format is not compatible with old tools.
Existing user dictionary will be automatically backed up and migrated
however old tools will not be able to manipulate the dictionary directly.
Use the new chewing-cli tool to inspect/dump/build user dictionaries.
Updated documentation will be provided in final release.

• Added

  • A new command line tool chewing-cli is included in this release. It can
    create or inspect dictionary files.
  • Reintroduce DictionaryMut as a separate trait.

• Changed

  • CMake default preset switched to build with rust.
  • CMake minimum required version bumped to 3.24.0.
  • Improved TrieBufDictionary::from ergonomics.
  • Rename DictEntries to Entries.
  • Ensure all fallible API returns Error.
  • Improved algorithm so using Tab key to find alternative phrasing is more
    responsive.
  • Easy symbol input now accepts arbitrary long "symbols". (rust)

• Removed

  • HashMap implemented Dictionary was removed.

• Documentation

  • Clarify the accepted value of chewing_handle_Default method.

Full Changelog: v0.7.0...v0.8.0-rc.1

v0.7.0

What's New in libchewing 0.7.0 (Apr 7, 2024)

This is the first beta release of the Rust rewrite. The library has been tested
with the AFL++ fuzzer and most of the obvious bugs have been fixed so it should
not crash for normal use. Now we invite power users to daily drive the beta
release in order to find more subtle bugs.

The Rust rewrite is ABI compatiable with previous releases. The most notable
difference with the C version includes a new portable dictionary format that can
be used both as system dictionary and user dictionary. Users with existing user
phrase dictionary will be automatically migrated. Old files will be backed up
automatically. With these change we will be able to support dictionary sharing
and loading multiple dictionaries in future releases.

• Future roadmap

  • 0.8.x after bug bashing this version will be the first public release with
    rust implementation as the default.
  • 0.9.x will focus on the stablization of the Rust crate API.
  • 1.0.0 will be the first public release of the Rust API. C implementation
    will be removed from the repository.

• Added

  • Added Colemak-DH ANSI/Orth layout support
  • Replace bespoke RIFF+TLV file format with standard DER format (rust)
  • Supporting migrating user dictionary file to new format. (rust)
  • Allow creating in memory user dictionary (rust)
  • New chewing-cli tool can be used to create or inspect dictionary files. (rust)
  • New chewing-cli tool can dump dictionary as or import from CSV file. (rust)
  • Audit dependencies using cargo-vet (rust)

• Changed

  • Buffer user dictionary changes and flush in background without blocking main
    thread (rust)

• Bug fixed

  • Some unsigned underflow issue found by afl++ (rust)
  • Skip symbol selection if preedit buffer is empty (rust)
  • Allow numlock in selection mode (rust)
  • Avoid infinite recursion (rust)
  • Ensure to return KeyBehavior::Commit when we push to commit buffer (rust)
  • Adjust selection offset after delete symbols (rust)
  • Disable trace logging if no logger is enabled (rust)
  • Should not start selection in English mode (rust)
  • Use offset to select from paginated phrase list (rust)

• Documentation

  • Added document for fuzzers (rust)
  • Describe how to compile dictionary files
  • Update build instruction and Rust MSRV info

• Testing

  • Added a fuzzer harness with afl++ (rust)

New Contributors

• @mlouielu made their first contribution in https://github.com/chewing/libchewing/pull/441

Full Changelog: v0.6.0...v0.7.0

v0.7.0-rc.2

[0.7.0-rc.2] - 2024-03-31

🐛 Bug Fixes

• (editor) Ensure to return KeyBehavior::Commit when we push to commit buffer
• (conv) Adjust selection offset after delete symbols
• (logging) Disable trace logging if no logger is enabled
• (editor) Should not start selection in English mode

🚜 Refactor

• Fix more clippy warnings

🧪 Testing

• (fuzzing) Disable checksum for fuzzing build
• (fuzzing) Test that pre-edit buffer does not suddenly disappear
• (fuzzing) Use more bits from input randomness

⚙️ Miscellaneous Tasks

• Bump pre-release version

Build

• Rename chewing-testhelper to chewing_testhelper
• (vet) Add cargo-vet configs
• (vet) First round of audits
• Change source archive to use zstd
• (vet) Another round of audits

v0.7.0-rc.1

What's New in libchewing 0.7.0-rc.1 (Mar 25, 2024)

This is the first beta release of the Rust rewrite. The library has been tested
with the AFL++ fuzzer and most of the obvious bugs have been fixed so it should
not crash for normal use. Now we invite power users to daily drive the beta
release in order to find more subtle bugs.

The Rust rewrite is ABI compatiable with previous releases. The most notable
difference with the C version includes a new portable system dictionary format
and two new user dictionary format. Users with existing user phrase dictionary
will be automatically migrated to either a new sqlite3 schema or CDB based
format depending on compile time configuration. Old files will be backed up
automatically. With these change we will be able to support dictionary sharing
and loading multiple dictionaries in future releases.

• Future roadmap

  • 0.8.x after bug bashing this version will be the first public release with
    rust implementation as the default.
  • 0.9.x will focus on the stablization of the Rust crate API.
  • 1.0.0 will be the first public release of the Rust API. C implementation
    will be removed from the repository.

• Added

  • Added Colemak-DH ANSI/Orth layout support
  • Supporting migrating user dictionary file to new format. (rust)
  • Allow creating in memory user dictionary (rust)
  • New chewing-cli tool can be used to create or inspect dictionary files. (rust)
  • New chewing-cli tool can dump dictionary as or import from CSV file. (rust)

• Changed

  • New trie dictionary format now has built-in checksum. (rust)

• Bug fixed

  • Some unsigned underflow issue found by afl++ (rust)
  • Skip symbol selection if preedit buffer is empty (rust)
  • Allow numlock in selection mode (rust)
  • Avoid infinite recursion (rust)

• Documentation

  • Added document for fuzzers (rust)
  • Describe how to compile dictionary files
  • Update build instruction and Rust MSRV info

• Testing

  • Added a fuzzer harness with afl++ (rust)

v0.6.0

What's New in libchewing 0.6.0 (Jan 27, 2024)

This release contains many improvements and bug fixes. It's the first
release since 2016. We have started a major rewrite in Rust so we expect
to have more frequent releases in the following months.

• Added

  • New API chewing_phone_to_bopomofo to convert syllable in u16 to bopomofo
    string [ShengYenPeng]. #166
  • New API chewing_set_autoLearn and chewing_get_autoLearn can be used to
    disable and enable new phrase auto learning. [Hiunn-hué] #261
  • Add default config test for autoLearn option [Billy Su].
  • Add Android platform support [hiroshiyui]. #408
  • New chewing Rust crate can be downloaded from
    crates.io [Kan-ru Chen]. #423
  • Add several CMake presets for supported configurations [Kan-ru Chen]. #424

• Changed

  • Implement has_userphrase in testhelper with the public
    chewing_userphrase_lookup API. [ShengYenPeng]. #223
  • Updated HSU layout's JVC conversion to be more intuitive [Hiunn-hué]. #259
  • Phrase auto learning now only triggers after committing [Hiunn-hué]. #260
  • chewing_handle_CtrlNum now handles cursor better and has clearer error
    message if there is nothing to be added to user phrase. [Hiunn-hué] #277
  • chewing_handle_ShiftLeft and chewing_handle_ShiftRight now also displays
    aux message for new user phrases added. [Hiunn-hué] #278
  • Symbol used for full-width colon changed from ︰ (U+FE30) to ： (U+FF1A)
    [school510587] #304
  • Support cross compiling with qemu. [sgn] #333
  • CMake minimum version changed to 3.21.0
  • Rewrite chewing internal in Rust [Kan-ru Chen] #347 #378

• Dictionary

  • Add new syllable ㄒㄩㄢ for 亘 [yan12125] #252 #253

• Removed

  • Autotools build tools are removed. CMake is the recommended way to build
    libchewing.

• Bug fixed

  • Fix ChoiceEndChoice expression always true [Billy Su].
  • Fix strncpy warning in hash.c [Bo-Cheng Jhan].
  • Fix unable to input some Bopomofos with ET26 layout [Hiunn-hué]. #235
  • Fix double free in userphrase-hash's TerminateUserphrase [cwlin]. #254
  • Fix hash item offsett calculation in uhash [cwlin]. #255
  • Fix ChoiceEndChoice expression always true [Billy Su]. #258
  • Fix clearing bopomofo buffer after switching Chi-Eng mode [Hiunn-hué]. #268
  • Fix chewing_handle_Space cannot input space with empty buffer [Hiunn-hué]. #274
  • Fix a strncpy warning and use ARRAY_SIZE [school510587] #296
  • Enhance chewing.py to support platforms with 64-bit pointer [cheese] #310
  • Fix init_database to correctly read phrase frequency for sorting [yan12125] #334 #337
  • Fix installation of data files [yan12125] #350
  • Fix building shared library on all supported platforms [Kan-ru Chen] #375 #376
  • Fix dvorak-qwerty symbol mapping [hiroshiyui] #407
  • prevent memory leak in error case of GetDefaultUserPhrasePath [Kuang-che Wu].
  • Fix PinYinData keySeq buffer overflow in PinYinInput() [Kuang-che Wu]. #238
  • Fix trying to allocate buffer of negative size [Kuang-che Wu].
  • Fix access violation due to negative cursor position [Kuang-che Wu].
  • Fix several buffer overflow. [ChangZhuo Chen]. #241 #242

• Platform-specific

  • Install instructions for macOS [Chocobo1].

New Contributors

• @cwlin made their first contribution in https://github.com/chewing/libchewing/pull/254
• @Billy4195 made their first contribution in https://github.com/chewing/libchewing/pull/258
• @splasky made their first contribution in https://github.com/chewing/libchewing/pull/280
• @school510587 made their first contribution in https://github.com/chewing/libchewing/pull/296
• @cheese made their first contribution in https://github.com/chewing/libchewing/pull/310
• @sgn made their first contribution in https://github.com/chewing/libchewing/pull/333
• @ianchen-tw made their first contribution in https://github.com/chewing/libchewing/pull/396
• @hiroshiyui made their first contribution in https://github.com/chewing/libchewing/pull/408

Full Changelog: v0.5.1...v0.6.0

v0.6.0-rc.2

What's New in libchewing 0.6.0-rc.2 (UNRELEASED)

• Added

  • Initial documentation to the Rust chewing crate.

• Changed

  • Many documentation improvements.

• Fixed

  • Build texinfo on macOS.

What's New in libchewing 0.6.0-rc.1 (UNRELEASED)

This release contains many improvements and bug fixes. It's the first
release since 2016. We have started a major rewrite in Rust so we expect
to have more frequent releases in the following months.

• Added

  • New API chewing_phone_to_bopomofo to convert syllable in u16 to bopomofo
    string [ShengYenPeng]. #166
  • New API chewing_set_autoLearn and chewing_get_autoLearn can be used to
    disable and enable new phrase auto learning. [Hiunn-hué] #261
  • Add default config test for autoLearn option [Billy Su].
  • Add Android platform support [hiroshiyui]. #408
  • New chewing Rust crate can be downloaded from
    crates.io [Kan-ru Chen]. #423
  • Add several CMake presets for supported configurations [Kan-ru Chen]. #424

• Changed

  • Implement has_userphrase in testhelper with the public
    chewing_userphrase_lookup API. [ShengYenPeng]. #223
  • Updated HSU layout's JVC conversion to be more intuitive [Hiunn-hué]. #259
  • Phrase auto learning now only triggers after committing [Hiunn-hué]. #260
  • chewing_handle_CtrlNum now handles cursor better and has clearer error
    message if there is nothing to be added to user phrase. [Hiunn-hué] #277
  • chewing_handle_ShiftLeft and chewing_handle_ShiftRight now also displays
    aux message for new user phrases added. [Hiunn-hué] #278
  • Symbol used for full-width colon changed from ︰ (U+FE30) to ： (U+FF1A)
    [school510587] #304
  • Support cross compiling with qemu. [sgn] #333
  • CMake minimum version changed to 3.21.0
  • Rewrite chewing internal in Rust [Kan-ru Chen] #347 #378

• Dictionary

  • Add new syllable ㄒㄩㄢ for 亘 [yan12125] #252 #253

• Bug fixed

  • Fix ChoiceEndChoice expression always true [Billy Su].
  • Fix strncpy warning in hash.c [Bo-Cheng Jhan].
  • Fix unable to input some Bopomofos with ET26 layout [Hiunn-hué]. #235
  • Fix double free in userphrase-hash's TerminateUserphrase [cwlin]. #254
  • Fix hash item offsett calculation in uhash [cwlin]. #255
  • Fix ChoiceEndChoice expression always true [Billy Su]. #258
  • Fix clearing bopomofo buffer after switching Chi-Eng mode [Hiunn-hué]. #268
  • Fix chewing_handle_Space cannot input space with empty buffer [Hiunn-hué]. #274
  • Fix a strncpy warning and use ARRAY_SIZE [school510587] #296
  • Enhance chewing.py to support platforms with 64-bit pointer [cheese] #310
  • Fix init_database to correctly read phrase frequency for sorting [yan12125] #334 #337
  • Fix installation of data files [yan12125] #350
  • Fix building shared library on all supported platforms [Kan-ru Chen] #375 #376
  • Fix dvorak-qwerty symbol mapping [hiroshiyui] #407
  • prevent memory leak in error case of GetDefaultUserPhrasePath [Kuang-che Wu].
  • Fix PinYinData keySeq buffer overflow in PinYinInput() [Kuang-che Wu]. #238
  • Fix trying to allocate buffer of negative size [Kuang-che Wu].
  • Fix access violation due to negative cursor position [Kuang-che Wu].
  • Fix several buffer overflow. [ChangZhuo Chen]. #241 #242

• Platform-specific

  • Install instructions for Mac OS [Chocobo1].

New Contributors

• @cwlin made their first contribution in https://github.com/chewing/libchewing/pull/254
• @Billy4195 made their first contribution in https://github.com/chewing/libchewing/pull/258
• @splasky made their first contribution in https://github.com/chewing/libchewing/pull/280
• @school510587 made their first contribution in https://github.com/chewing/libchewing/pull/296
• @cheese made their first contribution in https://github.com/chewing/libchewing/pull/310
• @sgn made their first contribution in https://github.com/chewing/libchewing/pull/333
• @ianchen-tw made their first contribution in https://github.com/chewing/libchewing/pull/396
• @hiroshiyui made their first contribution in https://github.com/chewing/libchewing/pull/408

Full Changelog: v0.5.1...v0.6.0-rc.2

v0.6.0-rc.1

What's New in libchewing 0.6.0-rc.1 (UNRELEASED)

This release contains many improvements and bug fixes. It's the first
release since 2016. We have started a major rewrite in Rust so we expect
to have more frequent releases in the following months.

• Added

  • New API chewing_phone_to_bopomofo to convert syllable in u16 to bopomofo
    string [ShengYenPeng]. #166
  • New API chewing_set_autoLearn and chewing_get_autoLearn can be used to
    disable and enable new phrase auto learning. [Hiunn-hué] #261
  • Add default config test for autoLearn option [Billy Su].
  • Add Android platform support [hiroshiyui]. #408
  • New chewing Rust crate can be downloaded from
    crates.io [Kan-ru Chen]. #423
  • Add several CMake presets for supported configurations [Kan-ru Chen]. #424

• Changed

  • Implement has_userphrase in testhelper with the public
    chewing_userphrase_lookup API. [ShengYenPeng]. #223
  • Updated HSU layout's JVC conversion to be more intuitive [Hiunn-hué]. #259
  • Phrase auto learning now only triggers after committing [Hiunn-hué]. #260
  • chewing_handle_CtrlNum now handles cursor better and has clearer error
    message if there is nothing to be added to user phrase. [Hiunn-hué] #277
  • chewing_handle_ShiftLeft and chewing_handle_ShiftRight now also displays
    aux message for new user phrases added. [Hiunn-hué] #278
  • Symbol used for full-width colon changed from ︰ (U+FE30) to ： (U+FF1A)
    [school510587] #304
  • Support cross compiling with qemu. [sgn] #333
  • CMake minimum version changed to 3.21.0
  • Rewrite chewing internal in Rust [Kan-ru Chen] #347 #378

• Dictionary

  • Add new syllable ㄒㄩㄢ for 亘 [yan12125] #252 #253

• Bug fixed

  • Fix ChoiceEndChoice expression always true [Billy Su].
  • Fix strncpy warning in hash.c [Bo-Cheng Jhan].
  • Fix unable to input some Bopomofos with ET26 layout [Hiunn-hué]. #235
  • Fix double free in userphrase-hash's TerminateUserphrase [cwlin]. #254
  • Fix hash item offsett calculation in uhash [cwlin]. #255
  • Fix ChoiceEndChoice expression always true [Billy Su]. #258
  • Fix clearing bopomofo buffer after switching Chi-Eng mode [Hiunn-hué]. #268
  • Fix chewing_handle_Space cannot input space with empty buffer [Hiunn-hué]. #274
  • Fix a strncpy warning and use ARRAY_SIZE [school510587] #296
  • Enhance chewing.py to support platforms with 64-bit pointer [cheese] #310
  • Fix init_database to correctly read phrase frequency for sorting [yan12125] #334 #337
  • Fix installation of data files [yan12125] #350
  • Fix building shared library on all supported platforms [Kan-ru Chen] #375 #376
  • Fix dvorak-qwerty symbol mapping [hiroshiyui] #407
  • prevent memory leak in error case of GetDefaultUserPhrasePath [Kuang-che Wu].
  • Fix PinYinData keySeq buffer overflow in PinYinInput() [Kuang-che Wu]. #238
  • Fix trying to allocate buffer of negative size [Kuang-che Wu].
  • Fix access violation due to negative cursor position [Kuang-che Wu].
  • Fix several buffer overflow. [ChangZhuo Chen]. #241 #242

• Platform-specific

  • Install instructions for Mac OS [Chocobo1].

New Contributors

• @cwlin made their first contribution in https://github.com/chewing/libchewing/pull/254
• @Billy4195 made their first contribution in https://github.com/chewing/libchewing/pull/258
• @splasky made their first contribution in https://github.com/chewing/libchewing/pull/280
• @school510587 made their first contribution in https://github.com/chewing/libchewing/pull/296
• @cheese made their first contribution in https://github.com/chewing/libchewing/pull/310
• @sgn made their first contribution in https://github.com/chewing/libchewing/pull/333
• @ianchen-tw made their first contribution in https://github.com/chewing/libchewing/pull/396
• @hiroshiyui made their first contribution in https://github.com/chewing/libchewing/pull/408

Full Changelog: v0.5.1...v0.6.0-rc.1

v0.5.1

What's New in libchewing 0.5.1 (May 18, 2016)

• Bug fixed:
  • Fix wrong CHEWING_DATADIR definition in CMake build #222.
  • Fix j, k selection when symbol in between #149 #221.

v0.5.0

What's New in libchewing 0.5.0 (May 2, 2016)

• New feature
  • Add Carpalx keyboard layout support #217.
• Dictionary
  • Update dictionary #151 #155 #188 #191 #201.
  • Add debug tool: dump_database.
  • Sort tsi.src #212.
• Misc
  • Tweak several comments.
  • Fix libchewing document.
  • Remove old/dead code.
  • Improve code readability and consistency.
  • Update automake syntax #114.
  • Add test cases #169 #177.
  • Update Python sample code.
  • Add debug tool dump_database.
  • Integrate coverity scan in https://scan.coverity.com/.
• Platform-specific
  • Update README for OS X.
  • Set WITH_INTERNAL_SQLITE3 to true by default in MS-Windows.
  • Fix Visual Studio compiling warnings/errors #168 #171 #189 #190 #213.
  • Add a cmake option BUILD_DLL to enable building *.dll with MS VC++ #185.
  • Make the python binding of libchewing support Windows dll files #186.
  • Don't use SIGSTOP in MS-Windows.
• Bug fixed
  • Check if ld supports --no-defined #144.
  • Fix unexpected candidate disorder for '1' #153
  • Replace bash-izm '==' operator with '=' for test(1) # 158.
  • Fix several Coverity errors #172 #173.
  • Fix ChewingKillChar which is disabling OpenSymbolChoice #160 #181.
  • ㄅ cannot overwrite ㄆ in HSU and ET26 #170 #183.
• Successful stories:
  • Rime Input Method Engine (RIME) takes partial dictionary from
    libchewing.
  • PIME (writing input methods for Windows easily with Python) supports
    Chewing IM by default.

v0.4.0

What's New in libchewing 0.4.0 (Apr 11, 2014)

• Add the following APIs:
  • Userphrase manipulation
    • chewing_userphrase_enumerate
    • chewing_userphrase_has_next
    • chewing_userphrase_get
    • chewing_userphrase_add
    • chewing_userphrase_remove
    • chewing_userphrase_lookup
  • Keyboardless action
    • chewing_cand_open
    • chewing_cand_close
    • chewing_cand_string_by_index_static
    • chewing_cand_choose_by_index
    • chewing_cand_list_first
    • chewing_cand_list_last
    • chewing_cand_list_has_next
    • chewing_cand_list_has_prev
    • chewing_cand_list_next
    • chewing_cand_list_prev
    • chewing_commit_preedit_buf
    • chewing_clean_preedit_buf
    • chewing_clean_bopomofo_buf
  • Bopomofo buffer
    • chewing_bopomofo_Check
    • chewing_bopomofo_String_static
  • Static API
    • chewing_aux_String_static
    • chewing_buffer_String_static
    • chewing_cand_String_static
    • chewing_commit_String_static
    • chewing_kbtype_String_static
  • Misc
    • chewing_new2
• Dictionary format:
  • Remove text data support
  • Introduce platform independent binary data representation
  • Move binary data to share/libchewing/*
• Userphrase format:
  • Support SQLite-based user phrase storage and manipulation
• New coding style:
  • Use scripts/indent.sh to change coding style
• Bug fixed:
  • Fix chewing_zuin_Check wrong return value #62
  • Fix numlock key cannot output number #45
  • Fix negative frequency problem #75
  • Fix cannot select candidate when symbol exists #79
  • Avoid clearing the buffer when Up arrow key is pressed and escCleanAllBuf is 1.
  • Fix cannot handle numlock 9 correctly #88
  • Fix invalid characters are committed when pre-edit buffer is full
  • Fix AutoLearnPhrase cannot fill correct wordSeq #108
  • Fix wrong auto learn when symbol in between #112
  • Fix missing rpl_malloc for cross compilation #118
  • Fix '\n' appear in symbol table #126
  • Reopen candidate list for symbols with down key #127 #135
  • Let up key close candidate list of symbols #131 #138
• Misc:
  • Add `make check' in cmake build
  • Change MAX_UTF8_SIZE to 4
  • Update official website URL
  • Use CreateFileA() instead of CreateFile() to stick with the ANSI API
  • Do not check phrase longer than MAX_PHRASE_LEN
  • Implement dynamic programming phrasing algorithm #73
  • Clean bopomofo buffer when mode change to English mode #89 #97
  • Update README to GFM.
  • Replace 'INCLUDE' with 'AM_CPPFLAGS' #68 #114 #116
  • Use [Shift]+[-] to input EM DASH (u+2014) as default #113 #124
  • Use Backspace to close candidate window #128
  • Use PgUp/PgDn to turn pages of candidates #129
  • Update Windows default search path #78
  • The word "zuin", "Zhuyin" are obsolete terms for "Bopomofo"
  • Add const whenever possible in several APIs
  • Toggle shape mode with shift-space #141 #142
• Successful stories:
  • chewing-windows-tsf project provides Text Services Framework
    support for Microsoft Windows.