Stetl up to and including version 1.3 only runs with Python 2 (2.7+). Starting with Stetl v2.0 only Python 3 (3.6+) will be supported. You may want to read Upgrade to Python 3 when upgrading from a Stetl pre-v2 version.
Easiest is to first install the Stetl-dependencies (see below) and then install and maintain Stetl on your system as a Python package (pip is preferred).
(sudo) pip install stetl or easy_install stetl
Alternatively you can download Stetl from Github: by cloning (preferred) or downloading: https://github.com/geopython/stetl/archive/master.zip and then install locally
(sudo) python setup.py install
Try the examples first. This should work on Linuxes and Mac OSX.
Windows installation may be more involved depending on your local Python setup. Platform-specific installations below.
You may also want to download the complete .tar.gz distro from PyPi: https://pypi.python.org/pypi/Stetl . This includes the examples and tests.
Thanks to Bas Couwenberg, work is performed to provide Stetl as Debian packages on both Debian and Ubuntu, see details:
https://packages.debian.org/search?keywords=stetl (Debian) and
https://launchpad.net/ubuntu/+source/python-stetl (Ubuntu, Xenial and later).
Stetl is split into 2 packages
python-stetl, the Python framework and
stetl the command line utility.
NB the versions of these packages may be older than when installing Stetl via pip from PyPi
or directly from GitHub. Always check this first.
Stetl depends on the following Python packages:
- GDAL (>=v2.4) bindings for Python
- psycopg2 (PostgreSQL client)
- lxml >=4.4.2
- Jinja2 templating
GDAL Python binding requires the native GDAL/OGR libs and tools to be installed.
lxml http://lxml.de/installation.html requires the native (C) libraries:
- libxslt (required by lxml)
- libxml2 with Python bindings (required by lxml)
When using the
Jinja2 templating filter,
Jinja2TemplatingFilter, see http://jinja.pocoo.org:
Deprecated is used to indicated deprecated functions and classes.
Platform-specific guidelines for dependencies follow next.
For Debian-based distro’s like Ubuntu and Debian itself, most packages should be able to be installed via apt-get.
Tip: to get latest versions of GDAL and other Open Source geospatial software, best is to add the UbuntuGIS Repository. Below a setup that works in Ubuntu 16.04 Xenial using Debian/Ubuntu packages. In some cases you may choose to install the same packages via pip3 to have more recent versions like for lxml.
More Linux Tips. See also:
- the install commands for Debian in the [Dockerfile](https://github.com/geopython/stetl/blob/master/Dockerfile).
- the install commands for Ubuntu in the [Travis file](https://github.com/geopython/stetl/blob/master/.travis.yml).
apt-get install python3-setuptools apt-get install python3-dev apt-get install python3-pip pip3 install --upgrade pip
libxml2/libxsltlibs are usually already installed. Together with Python
lxml, the total install for
apt-get install python3-libxml2 apt-get install python-libxslt1 apt-get install libxml2-dev libxslt1-dev lib32z1-dev apt-get install python3-lxml
GDAL(http://gdal.org) version 2+ with Python bindings:
# Add UbuntuGIS repo to get latest GDAL, at least v2 on Ubuntu 16.04, Xenial. add-apt-repository ppa:ubuntugis/ubuntugis-unstable apt-get update apt-get install gdal-bin gdalinfo --version # should show something like: GDAL 2.4.0, released 2019/03/04 apt-get install python3-gdal
the PostgreSQL client library for Python
apt-get install python3-psycopg2
apt-get install python3-jinja2
Dependencies can best be installed via Homebrew.
Tip: sometimes installing GDAL Python bindings can be tricky as the installed GDAL binaries must be compatible. To install the right version you may use:
pip install GDAL==`gdalinfo --version | cut -d' ' -f2 | cut -d',' -f1`
Best is to install GDAL and python using the OSGeo4W Installer from http://trac.osgeo.org/osgeo4w.
- Download and run the OSGeo4W Installer
- On the
Select Packagespage expand
Commandline_Utilitiesand Select from the list
easy_installto allow you to install
- Download the
- Open the OSGeo4W Shell (Start > Programs > OSGeo4W > OSGeo4W > OSGeo4W Shell)
- Change to the folder that you downloaded
ez_setup.pyto (if you downloaded to C:Temp then run cd C:Temp)
easy_installby running python
- To install
lxmlwith easy_install run
Only Psycopg2 needs explicit installation. Many install via: http://www.stickpeople.com/projects/python/win-psycopg. Once the above has been installed you should have everything required to run Stetl.
Alternatively you may use Portable GIS. Still you will need to manually install psycopg2. See http://www.archaeogeek.com/portable-gis.html for details.
If you installed via Python ‘pip’ you can check if you run the latest version
You should get meaningful output like
2013-09-16 18:25:12,093 util INFO running with lxml.etree, good! 2013-09-16 18:25:12,100 util INFO running with cStringIO, fabulous! 2013-09-16 18:25:12,122 main INFO Stetl version = 1.0.3 usage: stetl [-h] -c CONFIG_FILE [-s CONFIG_SECTION] [-a CONFIG_ARGS]
Especially check the Stetl version number. You can also use the -v or –version option for stetl.
Try running the examples when running with a downloaded distro.
cd examples/basics ./runall.sh
Look for any error messages in your output.
Run Unit Tests¶
You can run unit tests to completely verify your installation. First install some extra packages:
pip install -r requirements-dev.txt
Then run the tests using nose2.
Install with Docker¶
The fastest way to use Stetl is via Docker. The Stetl Docker Image is lightweight, compressed just over 100MB, based on a Debian “slim” Docker Image.
Your environment needs to be setup to use Docker and probably you want to use some tooling like Vagrant. The author uses a combination of VirtualBox with Ubuntu and Vagrant on Mac OSX to run Docker, but this is a bit out of scope here.
Assuming you have a working Docker environment, there are two ways to install Stetl with Docker:
- build a Docker image yourself using the Dockerfile in https://github.com/geopython/stetl/blob/master/Dockerfile
- use a prebuilt public Stetl Docker image from Docker Hub: https://hub.docker.com/r/geopython/stetl
When rebuilding you can add build arguments for your environment, defaults:
ARG TIMEZONE="Europe/Amsterdam" ARG LOCALE="en_US.UTF-8" ARG ADD_PYTHON_DEB_PACKAGES="" ARG ADD_PYTHON_PIP_PACKAGES=""
For example building with extra Python packages, building your local Docker Image:
docker build --build-arg ADD_PYTHON_DEB_PACKAGES="python-requests python-tz" -t geopython/stetl:local . docker build --build-arg ADD_PYTHON_PIP_PACKAGES="scikit-learn==0.18 influxdb" -t geopython/stetl:local .
Or you may extend the Stetl Dockerfile with your own Dockerfile.
For running Stetl using Docker see Using Docker.