Creating Python Packages

Creating Python Packages#

In this final chapter, we will introduce how to generate Python packages. Packages are a way to organize and distribute Python code. They can be used to share code with others, or to create reusable components for your own projects. For more on how to write, distribute and install Python packages, we recommend reading the Python Packaging User Guide. Here we will limit ourselves to a few general ideas that will get you started and also will be useful to understand packages written by others.

Directory structure#

To generate a Python package, you will need to create a directory structure and write a few files. The basic structure of a Python package is as follows:

my_package/
├── __init__.py
├── module1.py
├── module2.py
└── ...

Although it can be empty, the __init__.py file is essential for Python to recognize the directory as a package. It can also be used to import specific modules from the package. Each module can contain any type of Python code, such as functions, classes, and variables. For example, the following __init__.py file would import the module1 and module2 modules from the my_package package:

from .module1 import function1, class1
from .module2 import function2, class2

This would allow you to import these modules from anywhere in your code using the following syntax:

import my_package

# Call a function from the module1 module
my_package.function1()

# Create an instance of a class from the module2 module
my_package.class2()

If your package depends on other Python packages, you can specify these dependencies in the setup.py file. This will ensure that you have everything you need for your code to run. To specify a dependency, you can use the install_requires keyword in the setup.py file. For example, the following setup.py file would specify a dependency on the numpy package:

from setuptools import setup

setup(
    name="my_package",
    version="1.0.0",
    install_requires=["numpy"],
)

As you can see, here we are making use of the setuptools package. When your package is installed, numpy will also be installed automatically.

You can include many other things in your package. We recommend writing a README file, where you can describe what the package does, who are the developers and provide some installation instructions. Additionally, it is often useful to provide a LICENSE, so that it is clear to users which conditions apply to use, modify and redistribute the code. For example, below you can see the text that you should include if you want to use the short and simple permissive MIT license

MIT License

Copyright (c) [year] [fullname]

Permission is hereby granted, free of charge, to any person obtaining a copy
of this software and associated documentation files (the "Software"), to deal
in the Software without restriction, including without limitation the rights
to use, copy, modify, merge, publish, distribute, sublicense, and/or sell
copies of the Software, and to permit persons to whom the Software is
furnished to do so, subject to the following conditions:

The above copyright notice and this permission notice shall be included in all
copies or substantial portions of the Software.

THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR
IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY,
FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE
AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER
LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM,
OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE
SOFTWARE.

Deployment#

Once you have created your package structure and written your code, you can generate the distribution archives using the following command:

python setup.py sdist bdist_wheel

This will create two files in the dist/ directory:

my_package-1.0.0.tar.gz: This is the source distribution archive. It can be used to install the package on any system that has Python installed.
my_package-1.0.0-py3-none-any.whl: This is the wheel distribution archive. It is a pre-compiled binary archive that can be installed on any system that has Python 3 installed.

To upload your package to the Python Package Index (PyPI), you can use the following command:

twine upload dist/*

Here we are using the Twine utility for publishing the package. This command will upload the source and wheel distribution archives to PyPI. Once your package is uploaded, it can be installed by anyone using the following command:

pip install my_package

Packaging with conda#

In this course, we have been using the Anaconda Python distribution. Anaconda has its own package manager for Python. It can be used to install, manage, and update Python packages. To package a Python package with Conda, you will need to create a meta.yaml file. This file describes the package and its dependencies. The basic structure of a meta.yaml file is as follows:

package:
  name: my_package
  version: 1.0.0

build:
  number: 0

requirements:
  build:
    - python
    - setuptools
    - wheel

  run:
    - numpy

The package section specifies the name and version of the package. The build section specifies the build number of the package. This number is incremented each time the package is rebuilt. The requirements section specifies the dependencies of the package for either building or running it.

Once you have created a meta.yaml file, you can build the package using the following command:

conda build .

To install the package, you can use the following command:

conda install my_package

To publish a Conda package to Anaconda.org, you will need to install the anaconda-client package:

conda install anaconda-client

Once you have installed the anaconda-client package, you can log in to Anaconda.org using the following command:

anaconda login

To publish your package to Anaconda.org, you can use the following command:

anaconda upload my_package.tar.bz2

This will upload the Conda package to Anaconda.org. Once the package is uploaded, it can be installed by anyone using the following command:

conda install -c anaconda my_package

Testing your code#

It is important to test your package thoroughly before releasing it. You can do this by writing a test suite. A test suite is a collection of tests that are used to verify the functionality of your package.

You can use any Python testing framework to write a test suite. Some popular testing frameworks include unittest, pytest, and nose. unittest is the default Python testing framework. It is a simple and easy-to-use framework that provides a number of features for writing and running tests. To write a unittest test, you need to create a subclass of the unittest.TestCase class. Each test is defined as a method of the subclass. The name of the test method must start with the prefix test_.

Within the test method, you can use the various assertion methods provided by unittest to verify the expected behavior of your code. For example, the following test method uses the assertEqual() method to verify that the sum of two numbers is equal to a third number: import unittest

class MyTestCase(unittest.TestCase):
    def test_sum(self):
        self.assertEqual(1 + 2, 3)

if __name__ == '__main__':
    unittest.main()

Once you have written a test suite, you can run it using the following command:

python -m unittest discover

This will run all of the tests in your test suite. If any of the tests fail, the command will exit with a non-zero status code.

You can calculate the test coverage for your package using the following command:

python -m coverage run -m unittest discover
python -m coverage report

This will generate a report that shows how much of your code is covered by your tests.