Intro
Want to run both python2 and python3 packages at the same time? Maybe you don't want to upgrade all your python2 services at the same time? With some small changes to the "ncs-python-start-vm" script you can. While you are at it maybe you would like to start using virtual environments in python? With a few more lines in "ncs-python-start-vm" you can have that too.
After the script changes all you need to do to run some packages with python3 is to put the package name in to the python3packages.txt file. If the package name isn't in the file it will just use the default python version.
If "ncs-python-start-vm" finds a virtual environment in your NSO package (in reality looking for the venv activate command) it will activate that environment first before starting the python vm.
Installation
If you are running a local installation (in a system install you have to change the paths below) you just have to create a bin folder in you NSO runtime folder and create your own copy of ncs-start-python-vm.
cd your_nso_runtime/
mkdir bin
cp $NCS_DIR/bin/ncs-start-python-vm bin/
touch bin/python3packages.txt
Modify your copy of ncs-python-start-vm script to the below or even better get it from the git repo (you might have to add the execute bit to the script, chmod +x ncs-python-start-vm)
#!/bin/sh
pypath="${NCS_DIR}/src/ncs/pyapi"
# Make sure everyone finds the NCS Python libraries at startup
if [ "x$PYTHONPATH" != "x" ]; then
PYTHONPATH=${pypath}:$PYTHONPATH
else
PYTHONPATH=${pypath}
fi
export PYTHONPATH
main="${pypath}/ncs_pyvm/startup.py"
echo "Starting ${main} $*"
# input argument 6 is the package name
# activating the virtual environment for the package
for path in ${PYTHONPATH//:/ }; do
basename=$(basename $path)
package=$(basename $(dirname $path))
packagepath=$(dirname $path)
#locate the python package path in $PYTHONPATH and check if it has a virtualenv
if [ "$basename" == "python" ] && [ "$package" == "$6" ] && [ -f "$packagepath/env/bin/activate" ]; then
echo "Found virtualenv for package $6, activating it"
source $packagepath/env/bin/activate
fi
done
# bin/python3packages.txt should be in a bin folder in the NSO runtime folder
if [ ! -z $(grep $6 "bin/python3packages.txt") ]; then
echo "Starting python v3 vm for package $6"
exec python3 -u ${main} $*
else
echo "Starting the system default python vm for package $6"
exec python -u ${main} $*
fi
Then change your ncs.conf so that it executes your new ncs-start-python-vm script instead of the default one.
vi ncs.conf
....
<python-vm>
<start-command>bin/ncs-start-python-vm</start-command>
</python-vm>
....
After that just add the packages you want to be running with python3 to the python3packages.txt file
cat python3packages.txt
external-id-allocator
selftest
l3vpn
Restart NSO, the restart is only needed for NSO to read in the new python vm startup script, might even be enough with a ncs --reload. Adding new packages to python3packages.txt will not require a reload.
ncs --stop
ncs
Using virtualenv
Installation
virtualenv is used to manage Python packages for different projects. Using virtualenv allows you to avoid installing Python packages globally which could break system tools or other projects. You can install virtualenv using pip.
python3 -m pip install --user virtualenv
Creating a virtual environment
venv (for Python 3) and virtualenv (for Python 2) allow you to manage separate package installations for different projects. They essentially allow you to create a “virtual” isolated Python installation and install packages into that virtual installation. When you switch projects, you can simply create a new virtual environment and not have to worry about breaking the packages installed in the other environments. It is always recommended to use a virtual environment while developing Python applications.
To create a virtual environment for you NSO package, go to your package directory and run venv. If you are using Python 2, replace venv with virtualenv in the below commands.
cd packages/l3vpn
python3 -m venv env
Activating a virtual environment
Before you can start installing or using packages in your virtual environment you’ll need to activate it. Activating a virtual environment will put the virtual environment-specific python and pip executables into your shell’s PATH.
if you want to switch projects or otherwise leave your virtual environment, simply run:
Installing packages in your virtualenv
Now that you’re in your virtual environment you can install packages. Let’s install the excellent Requests library from the Python Package Index (PyPI):
pip install requests
Using requirements files
Instead of installing packages individually, pip allows you to declare all dependencies in a Requirements File. For example you could create a requirements.txt file containing:
requests==2.18.4
google-auth==1.1.0
And tell pip to install all of the packages in this file using the -r flag:
pip install -r requirements.txt
Pip can export a list of all installed packages and their versions using the freeze command:
Which will output a list of package specifiers such as:
cachetools==2.0.1
certifi==2017.7.27.1
chardet==3.0.4
google-auth==1.1.1
idna==2.6
pyasn1==0.3.6
pyasn1-modules==0.1.4
requests==2.18.4
rsa==3.4.2
six==1.11.0
urllib3==1.22
This is useful for creating Requirements Files that can re-create the exact versions of all packages installed in an environment.
