Skip to content

Contribute to infinity_v3

New covered contributions are offered under the Infinity Restricted MIT License 1.0, unless different terms are agreed before inclusion. Contributors retain ownership and grant only rights they own or are authorized to license. Preserve inherited and third-party notices; independently licensed material retains its applicable terms. See LICENSING.md and the license scope.

Everyone is welcome to contribute, and we value everybody's contribution. Code contributions are not the only way to help the community. Answering questions, helping others, and improving the documentation are also immensely valuable.

It also helps us if you spread the word! Reference the library in blog posts about the awesome projects it made possible, shout out on Twitter every time it has helped you, or simply ⭐️ the repository to say thank you.

However you choose to contribute, please be mindful and respect our code of conduct.

This guide was heavily inspired by the awesome scikit-learn guide to contributing.

Working in the Infinity monorepo

The Rust project lives in libs/infinity-v3, beside the existing Python server and client. Run Rust commands from that directory so Cargo also loads its .cargo/config.toml and pinned rust-toolchain.toml:

cd libs/infinity-v3
cargo build --locked
cargo build --locked --no-default-features --features candle,http
cargo test --locked --no-run
cargo test --locked --lib
cargo fmt --all -- --check
cargo clippy --locked --lib --bins
cargo run --locked --bin infinity_v3 -- --help

CPU builds require a C/C++ compiler, pkg-config, OpenSSL development headers, and protobuf-compiler. On Debian or Ubuntu, install build-essential pkg-config libssl-dev protobuf-compiler. CUDA and LibTorch builds additionally require their SDKs; LibTorch is an optional workspace member and requires LIBTORCH. The default workspace builds the Candle CPU server without either SDK.

To run the optional pre-commit hooks, invoke them from the repository root:

pre-commit run --config libs/infinity-v3/.pre-commit-config.yaml --all-files

From the repository root, build the CPU image with the V3 directory as the Docker context:

docker build -f libs/infinity-v3/Dockerfile libs/infinity-v3

Use the same context for Dockerfile-cuda and Dockerfile-cuda-all. Run push-docker-images.sh from libs/infinity-v3 for local multi-variant builds.

V3 CI and Docker workflows live in the repository's root .github/workflows, with V3 path filters and independent names. CPU unit tests run on pull requests without model downloads or GPU credentials. Docker smoke tests download a pinned MiniLM checkpoint and run CPU inference before publishing. Checkpoint-dependent integration tests can be run separately with their documented fixtures. Docker pull requests build the CPU image without publishing. Main and v3.* or 3.* tags build the CPU and native NVIDIA images; manual runs can select additional variants. Publishing uses GHCR and the workflow token, with GitHub Actions build caching. Python release workflows remain scoped to 0.* and v0.* releases, and use the triggering release tag independently of V3 tags.

Releasing v3

  1. Update the workspace version in Cargo.toml, its local packages in Cargo.lock, the checked-in OpenAPI version, and the local publishing script. A stable release tag must match the Cargo version exactly.
  2. Merge the release changes and push a tag such as v3.0.0. The Docker workflow builds, checks version reporting and runs MiniLM CPU inference before pushing the cpu-* and nvidia-* images. GPU execution needs separate validation on the built NVIDIA image; the CI inference check runs without a GPU.
  3. After validating the images, publish the GitHub release. Stable v3 releases publish versioned documentation and update the website's latest alias. A tag push alone builds images; publishing the release publishes its docs.

Stable image tags update *-latest; main builds update *-main. Prerelease tags do not move stable aliases. The published CPU image uses Candle; the native LibTorch model catalog applies to the NVIDIA image and LibTorch source builds. Python/v2 releases update latest-v2 documentation independently of v3.

Ways to contribute

There are several ways you can contribute to infinity_v3.

  • Fix outstanding issues with the existing code.
  • Submit issues related to bugs or desired new features.
  • Contribute to the examples or to the documentation.

All contributions are equally valuable to the community. 🥰

Fixing outstanding issues

If you notice an issue with the existing code and have a fix in mind, feel free to start contributing and open a Pull Request!

Do your best to follow these guidelines when submitting a bug-related issue or a feature request. It will make it easier for us to come back to you quickly and with good feedback.

Did you find a bug?

The infinity_v3 solution is robust and reliable thanks to users who report the problems they encounter.

Before you report an issue, we would really appreciate it if you could make sure the bug was not already reported (use the search bar on GitHub under Issues). Your issue should also be related to bugs in the library itself, and not your code.

Once you've confirmed the bug hasn't already been reported, please include the following information in your issue so we can quickly resolve it:

  • Your OS type and version, as well as your environment versions (versions of rust, python, and dependencies).
  • A short, self-contained, code snippet that allows us to reproduce the bug.
  • The full traceback if an exception is raised.
  • Attach any other additional information, like screenshots, you think may help.

Do you want a new feature?

If there is a new feature you'd like to see in infinity_v3, please open an issue and describe:

  1. What is the motivation behind this feature? Is it related to a problem or frustration with the library? Is it a feature related to something you need for a project? Is it something you worked on and think it could benefit the community?

Whatever it is, we'd love to hear about it!

  1. Describe your requested feature in as much detail as possible. The more you can tell us about it, the better we'll be able to help you.
  2. Provide a code snippet that demonstrates the feature's usage.
  3. If the feature is related to a paper, please include a link.

If your issue is well written we're already 80% of the way there by the time you create it.

We have added templates to help you get started with your issue.

Do you want to implement a new model?

New models are constantly released and if you want to implement a new model, please provide the following information:

  • A short description of the model and a link to the paper.
  • Link to the implementation if it is open-sourced.
  • Link to the model weights if they are available.

If you are willing to contribute the model yourself, let us know so we can help you add it to infinity_v3!

Do you want to add documentation?

We're always looking for improvements to the documentation that make it more clear and accurate. Please let us know how the documentation can be improved such as typos and any content that is missing, unclear or inaccurate. We'll be happy to make the changes or help you make a contribution if you're interested!

I want to become a maintainer of the project. How do I get there?

infinity_v3 is a fork of Hugging Face's Text Embeddings Inference, maintained by this repository's maintainers. If you are interested in helping maintain the fork, open an issue in this repository to discuss how you would like to contribute.