Last Updated: February 25, 2016
·
733
· rochacon

Faster install of Python dependencies

When you want to use a specific revision or a private dependency in Python, usually you'll have something like this in your requirements.txt:

-e git+https://github.com/rochacon/project.git@revision#egg=project-rev

When Pip find an editable dependency, it will retrieve that repository using the specified VCS and authenticating with SSH, checkout to the given revision (or default branch) and then run the setup.py script. When running a requirements update Pip will again try to retrieve the repo to fetch updates.

This is not fast and for production use, usually, you don't need all this. But there is a simple way to make things a little quickier: use the source download feature of your VCS (here, we will use GitHub, but I'm confident that BitBucket and other services will have some similar feature).

The same dependency would look like:

https://github.com/rochacon/project/archive/revision.tar.gz#egg=project-rev

Now, pip only download a tarball and execute it's setup.py script, similar to how it would retrieve dependencies from PyPI.

For private repos, you can use Basic Auth for authentication, like:

https://user:pass@github.com/rochacon/project/archive/revision.tar.gz#egg=project-rev

I recommend you to create a company-pip GitHub account with read-only access exclusively for this, since those credentials will end up versioned in your project's requirements file.