HuggingFace datasets 是一个轻量级的数据集框架,用于数据集的加载、保存、预处理等。
pip install datasets
Successfully installed aiohttp-3.8.6 aiosignal-1.3.1 async-timeout-4.0.3 asynctest-0.13.0 attrs-24.2.0 datasets-2.13.2 frozenlist-1.3.3 multidict-6.0.5 multiprocess-0.70.14 pyarrow-12.0.1 xxhash-3.5.0 yarl-1.9.4
WARNING: Running pip as the 'root' user can result in broken permissions and conflicting behaviour with the system package manager. It is recommended to use a virtual environment instead: https://pip.pypa.io/warnings/venv
Note: you may need to restart the kernel to use updated packages.