Creating a local python pip repo
You can install it in any of your conda environments by doing: Where, recall, the X is the version so, once you’ve placed more versions in your channel, you can install specific versions. (the reason being that we’ll be using Python 2.7 for at least one more year and fear of packages or older versions being removed) I am looking at
Creating a local python pip repo
I am tasked to create and use a local PIP repo.
(the reason being that we’ll be using Python 2.7 for at least one more year and fear of packages or older versions being removed)
I am looking at bandersnatch and it is not clear to me whether it is an on-line mirroring tool which i need to run as a service, or can be used to offload a one-off copy?
I’d prefer a second option (don’t want to complicate the system unnecessarily), and would be satisfied by running an update say daily or even weekly.
An alternative approach would be to download only the packages and version we actually use by looking at the requirements.txt file, but this would require running an update every time a developer wants to add or update a package.
A way to create a local python package repository is throught Sonatype Nexus, with Nexus you can create some kinds of repos:
- Hosted repo (our own and internal repo)
- Proxy repo (proxy others repo)
- Group repo (group and priority sort a list of hosted and proxied repos)
For example, you can create a group repo with the following logic order: — First search the package in my own repo — If it not exists, search it on global public repo.
It is a transparent way to your app.
There is a Docker image if you want too. https://hub.docker.com/r/sonatype/nexus3
I used it before to different purposes and I see it very mature and complete.
a script that generates a simple repository with N recent versions of 4000 most used packages on pypi. Advantage is it can hold multiple versions as in pypi. https://gist.github.com/harisankar-krishna-swamy/cac5d1e6c1ae074b39286c1336bff63d
Setting up a local PyPi server with custom set of packages, I want to set up a local PyPi server with a custom set of packages and all their dependencies. What I have right now is a list of packages in format: …
How to clone PyPI repository and maintain as local repo
We have downloaded anaconda packages from https://repo.anaconda.com/pkgs/main/linux-64 to our local system and configured it as local conda repository. This is used to install conda packages on machines which dont have access to internet.
Same way would like to get all packages from PyPI and maintain a local repo. (some packages are only available in PyPI and not in anaconda).
Is there any repository from PyPI to download all the packages ?
There are several options, see: https://packaging.python.org/guides/index-mirrors-and-caches/
Python — How to use a local repository with pip install for, While building I have no connection to PyPi.python.org and I don’t have a local proxy available. If I could control easy_install I could try to point it to …
How to roll my own pypi?
I would like to run my own internal pypi server, for egg distribution within my organization.
I have found a few projects, such as:
As I understand it, pypi.python.org uses software called Cheese Shop.
- Why can’t I use cheeseshop itself? (I can’t find it, not sure it exists)
- How do other people solve this problem? (Currently we use blush svn to distribute eggs)
*edit: This seems canonical http://wiki.python.org/moin/PyPiImplementations. Still, I’m interested in feedback.
For light-weight solution, use pypiserver.
Update: PyPi is now powered by Warehouse, which is the replacement for Cheese Shop.
The source to Cheese Shop can be downloaded from https://bitbucket.org/pypa/pypi/src. There is also an example, from the page you linked to, of using Apache as a «dumb» python package repository:
# Mount pypi repositories into URI space Alias /pypi /var/pypi # /pypi/dev: Redirect for unknown packages (fallback to pypi) RewriteCond /var/pypi/dev/$1 !-d RewriteCond /var/pypi/dev/$1 !-f RewriteRule ^/pypi/dev/([^/]+)/?$ http://pypi.python.org/pypi/$1/ [R,L] RewriteCond /var/pypi/dev/$1/$2 !-f RewriteRule ^/pypi/dev/([^/]+)/([^/]+)$ http://pypi.python.org/pypi/$1/$2 [R,L] # /pypi/stable: Redirect for unknown packages (fallback to pypi) RewriteCond /var/pypi/stable/$1 !-d RewriteCond /var/pypi/stable/$1 !-f RewriteRule ^/pypi/stable/([^/]+)/?$ http://pypi.python.org/pypi/$1/ [R,L] RewriteCond /var/pypi/stable/$1/$2 !-f RewriteRule ^/pypi/stable/([^/]+)/([^/]+)$ http://pypi.python.org/pypi/$1/$2 [R,L] Warehouse
Warehouse would be your best bet in 2017. From the project’s README:
Warehouse is a next generation Python Package Repository designed to replace the legacy code base that currently powers PyPI
You can run Warehouse locally using docker and docker-compose . See Getting started in the documentation for instructions on how to set it up.
It is maintained by The Python Packaging Authority (PyPA) who work in cooperation with members of the Python core development team, and there is a live version running at https://pypi.org/ which mirrors everything in the legacy PyPI (https://pypi.python.org/).
devpi
We are using it in a corporate environment and are pretty satisfied. It supports replication, private indexes and index inheritance.
Disable authentication when uploading to local pypi server, I would recommend using devpi-server.Related command devpi allows entering user credentials via login subcomamand. The login is then valid for 10 hours and …
How can I host my own private conda repository?
I have a few python projects that are dependent on each other. I have different release versions for each project and different projects might be dependent on different release versions of a particular project. I would like to create my own conda repository on an internal server where I can push the releases of these projects as conda packages and the other projects can install the required version from there. Is this possible? If so how?
You can use a conda custom channel as your private repo. The essential steps are to use «conda build» to create a conda package, then copy that package into your custom channel (a directory), and now run conda index on that directory. You can then install packages from this channel by using the «conda install -c «.
An example, in more detail, let’s assume linux-64:
- Create the channel:
mkdir -p /tmp/my-conda-channel/linux-64 - Now assuming you have some project named «abc» with a meta.yaml and build.sh with some version X. Now you build it: conda build abc
- This will build a tar.bz2 file in your conda-bld directory. For example: ~/miniconda3/conda-bld/linux-64/abc-X-py35_0.tar.bz2. Copy that file to your channel: cp ~/miniconda3/conda-bld/linux-64/abc-X-py35_0.tar.bz2 /tmp/my-conda-channel/linux-64/
- Now index it: conda index /tmp/my-conda-channel/linux-64/
You’ve now uploaded that package to your custom channel. You can install it in any of your conda environments by doing:
conda install -c file://tmp/my-conda-channel/ abc=X Where, recall, the X is the version so, once you’ve placed more versions in your channel, you can install specific versions.
If you have a project that depends on the X version of «abc» then we simply add it to that projects meta.yaml. Example:
package: name: some-other-project version: 0.1 requirements: build: - abc X . Once you have created this channel it’s probably a good idea to add it to your .condarc file so that it will get automatically searched. For example:
channels: - file://tmp/my-conda-channel/ - defaults There are two parts to this: how to create the channel and how to use it. Part two is the hardest to do nicely.
The first part is described in detail in the conda documentation. You can serve the channel directly from files, or via a static webserver.
To use the channel, one approach is the -c file://tmp/my-conda-channel/ , but recent conda versions allow a much better solution via custom channels which where (recently?) added to conda.
The documentation is available via conda config —describe which includes this part:
# custom_channels (map: str) # A map of key-value pairs where the key is a channel name and the value # is a channel location. Channels defined here override the default # 'channel_alias' value. The channel name (key) is not included in the # channel location (value). For example, to override the location of # the 'conda-forge' channel where the url to repodata is # https://anaconda-repo.dev/packages/conda-forge/linux-64/repodata.json, # add an entry 'conda-forge: https://anaconda-repo.dev/packages'. # # custom_channels: <> The syntax for adding a channel is not documented, but reading the source the correct invocation is seen to be:
conda config --set custom_channels.my-conda-channel file://tmp/ (Note: my-conda-channel/ is not part of path). With this added to your config, you can now use your own channel in the same way you would conda-forge or other ‘built-in’ channels:
conda install -c my-conda-channel my-cool-package For anyone in an MS Windows setting, the correct set of slashes and backslashes for using this with a windows share is file://\\SOMECORP\Corp\conda\channels\ . Works a charm.
If you want to add the channel on Windows , try:
conda config --append channels file:///C:\tmp\my-conda-channel Make sure, you followed the instructions from the Paul’s and Janus’ answer.
PyPI, The Python Package Index (PyPI) is a repository of software for the Python programming language. PyPI helps you find and install software developed and …