Open source GIS introduced in this book

The open source GIS software and programs are numerous, and the system is huge. This book cannot explain it one by one. Even the subject of this book (Python and open source GIS) can only choose software and class libraries that are currently mature and widely used. This book focuses on how to use Python as a programming language for data processing. The following mainly use some of the class libraries.

• GDAL/OGR for reading and writing of raster and vector data;

• Proj.4 for map projection processing;

• Shapely, for spatial analysis of data;

• SpatiaLite is a small spatial database.

• Mapnik is used for cartography.

• Basemap is another set of cartographic tools.

• Other libraries, including pyshp, geojson, Descarts, GeoPandas, Folium, will also be introduced briefly in this book.

Installation and Configuration of Debian Linux

Perhaps the first problem in learning to use open source GIS is the use of operating systems. A large number of open source software are developed under GNU/Linux platform. Although many of them have been transplanted to Windows system, many of them are still difficult to install. In the field of GIS, there will be more problems.

In order to make better use of open source GIS tools, this book recommends using Debian/Ubuntu Linux operating system for learning. Debian can use Debian Stretch (Debian 9) released in 2017, and Ubuntu can use Ubuntu Bionic Beaver (Ubuntu 18.04) released in 2018. There is little difference between the two operating system versions, and there is no big difference in operation. All the code in this book has run successfully in both systems. But there are still some differences between the versions of the class libraries in the two systems, especially some newer ones. The results of code running are inconsistent. This book uses Debian Stretch in the writing process, using the Linux kernel 4.9. The operation and instructions in the book are also based on this system.

Debian is a voluntary organization dedicated to free software development and advocating the concept of the Free Software Foundation. Debian was founded in 1993 when Ian Murdock sent an open letter inviting software developers to participate in building a complete software distribution based on the newer Linux kernel. Over the years, this group of fans funded by the Free Software Foundation and influenced by the GNU concept has evolved into an organization with more than 1,000 Debian developers.

Check out the latest version of Debian 9:

$ uname -a
Linux v 4.9.0-6-amd64 #1 SMP Debian 4.9.82-1+deb9u3 (2018-03-02) x86_64 GNU/Linux

Installing related components and class libraries

Debian is the master version of Linux distributions such as Ubuntu, Linux Mint and Elementary OS. It has a robust package management system. Every component and application of Debian is built into the package installed in the system. Debian uses a set of tools called Advanced Packaging Tool (APT) to manage this package system.

In Debian-based Linux distributions, there are various tools that can interact with APT to facilitate user installation, deletion and management of software packages. Apt-get is one of the most popular command-line tools, and aptitude is a command-line gadget that combines GUI with aptitude.

Apt command is a newer command. The first stable version of the apt command was released in 2014, and the Ubuntu 16.04 system was released in 2016. The introduction of apt commands is to solve the problem of over-decentralization of commands. It includes the most widely used functional options since the emergence of apt-get commands, as well as functions rarely used in apt-cache and apt-config commands.

At present, Linux distributors are recommending apt command line tools. The apt command option is less memorable and therefore easier to use, and more importantly it provides the necessary options for Linux package management. Apt-get is not abandoned, but as an ordinary user, apt should be used first.

Python is widely used at present. The newer Linux distributions all have Python programs. All the functions described in this book were tested successfully in Python 3. Using Python 2 doesn’t make much difference.

Viewing the versions of software, class libraries, and their dependencies, you can use the aptcommand, which has different options:

apt show gdal-bin
apt depends gdal-bin

See the figure for component dependencies between class libraries.

Open Source GIS Class Library Dependency

In Debian system, install the tools introduced in this book, you can use the following commands:

apt install python3 python3-gdal gdal-bin \
    python3-pyproj proj-bin python3-shapely \
    fiona python3-fiona \
    python3-mapnik libspatialite7 \
    libsqlite3-mod-spatialite spatialite-bin \
    python3-mpltoolkits.basemap \
    python3-geopandas  python3-nose \
    python3-pygraphviz

In addition to the software tools already packaged in Debian, there are some Python tools that cannot be installed through the apt command. Python provides its own package installation tool, pip. Pip is a modern and universal Python package management tool, which provides the functions of finding, downloading, installing and uninstalling Python packages. It is currently a package installation tool recommended by PyPA (Python Packaging Authority). In addition to pip, software management tools in Python include easy install, setuptools, and distribute.

Use of Virtual Machines

Most readers may not have used Linux, let alone have their own experimental environment. But now that computer technology is so developed, this problem can easily be solved by technology, without having to reinvest in new equipment.

The following will introduce virtualization technology and virtual machine software. These technologies are not only used in the study of books, but also helpful in real life and development.

Editor and IDE

Writing code for development requires an interactive environment with the computer. These environments are mainly divided into editors and IDEs, there are many choices, here according to the author used some brief introduction.

Editor is a tool to assist data input to computer. General code editor is further able to support code highlighting, automatic indentation, automatic completion and other functions. There are also code editors that can execute and debug code, as well as support interaction with version control software. Editors are generally not limited to a particular language, but more general.

IDE (Integrated Development Environment) is an application program for the provider development environment. Besides the function of code editor, it integrates the common tools used in the development of code analysis, compilation, debugging, version control tools and so on. IDE is usually used for a particular language.

Code editors are lighter and faster than IDEs, but there are far fewer built-in tools. It’s more convenient to use a code editor when writing simple lines of code; but when the code becomes more and more complex, choosing an IDE can significantly improve the coding efficiency. In addition, it is also necessary to master one or more general code editors because of the convenience of installing code editors and the fact that IDEs may not be available due to environmental constraints.

Using Python language for development, you can use a general editor, or IDE (Integrated Development Environment), according to the situation of free choice. Python itself also provides IDLE (integrated development environment) integrated development environment for use.

Under Windows, you can choose free NotePad ++, Atom, Visual Studio Code, GVim (Graphical Vim), or Sublime 3; under Linux, you can use the Linux version of these software (if any), as well as VIM, GEdit and so on. Vim / GVim has a slightly longer learning cycle and takes time to get used to the way it is used, but it is more common in Linux environments.

On the IDE side, the most common open source tool is Eclipse, which can be used for Python development in conjunction with PyDev plug-ins, and commercial PyCharm. Both tools are developed in Java and can be used across platforms.

PyCharm is an integrated development environment developed by JetBrains, which is now widely welcomed by Python developers. Here is a special recommendation. It has a set of functions and tools to improve efficiency when developing in Python language, including code debugging, grammar highlighting, project management, code jumping, intelligent prompts, automatic completion, unit testing, version control, etc.

If Web development is not involved, the PyCharm Community version is free and can be downloaded from the official JetBrains website for installation (Java runtime environment support is required). In addition, for educators, or in maintaining open source projects (active projects for more than one year), you can apply for a free license to use the professional version. The PyCharm Professional Edition provides more features, such as installing the plugin to support the Latex language. Text editing and code debugging in the middle and late stages of this book is done in such an environment.

The authorization of PyCharm Professional Edition is applied through the open source Web CMS framework TorCMS