Uploading Datasets#

Create a dataset repository, organize its files by split or release, and publish the card with the data.

1mega repos create OWNER/DATASET --type dataset --private2mega datasets upload OWNER/DATASET ./dataset . --commit-message "Publish dataset"

For a large tree, use mega upload-large-folder. Keep data files, schema, checksums, and documentation in the same revision. Prefer a private repository until distribution rights and review are complete.

The huggingface_hub client can use the same repository contract:

1from huggingface_hub import HfApi2 3api = HfApi(endpoint="https://mega.tensorplay.cn", token="YOUR_MEGA_TOKEN")4api.create_repo("OWNER/DATASET", repo_type="dataset", exist_ok=True)5api.upload_folder(6    repo_id="OWNER/DATASET",7    repo_type="dataset",8    folder_path="./dataset",9    commit_message="Publish dataset",10)

For the corresponding hf CLI commands and compatibility limits, see Hugging Face Compatibility.

See Large Files and Dataset Repositories.