Installation

Local Installation

You can install the package from source after cloning the repository

git clone https://github.com/mhagdorn/taskfarm.git
cd taskfarm
python3 setup.py install

or simply using pip

pip install taskfarm

Setup

After installing the python package you need to connect to a database. For testing purposes you can use sqlite. However, sqlite does not allow row locking so if you use parallel workers a single task may get assigned to multiple workers. For production use you should use a postgres database instead.

You can set the environment variable DATABASE_URL to configure the database connection (see the SQLAlchemy documentation). For example

export DATABASE_URL=sqlite:///app.db

or

export DATABASE_URL=postgresql://user:pw@host/db

You then need to create the tables by running

adminTF --init-db

You can then create some users

adminTF -u some_user -p some_password

These users are used by the worker to connect to the service.

Running the Taskfarm Server

The taskfarm server is a flask web application. For testing you can run it locally using

export FLASK_APP=taskfarm
flask run

You can check the service is running by browsing to http://localhost:5000/ or running

curl http://localhost:5000/

For a production setup you need to deploy the flask application using a WSGI server such as gunicorn. The flask documentation lists the various options for self-hosting or hosting in the cloud a flask application.

Containerised Installation

Instead of installing the taskfarm server locally and managing the flask webapplication service you can run the taskfarm server as a containerised service. You need a working docker setup and docker compose. The taskfarm service is built using Ubuntu containers, one for the web application, one for the postgres database and one for the web server. You can build and start the containers using

docker-compose build

You need to initialise the database and create a user, ie

docker-compose run web  adminTF --init-db
docker-compose run web  adminTF -u taskfarm -p hello

You can now start the service

docker-compose up -d

and you can reach the taskfarm server on port 80. You can check the service is running by browsing to http://localhost/ or running

curl http://localhost/