Salt, storing Pillar data in MongoDB

Introduction

In Salt, the pillar interface offers a way to centrally-manage a tree of global variables presented to all or subset of minions. These variables can later be referenced in state files for configuration management. By default, this pillar data tree is stored as YAML flat files in /srv/pillar on the Salt master.

Using the External Pillars functionnality, it is also possible to retrieve pillars from other hierarchical data sources, like JSON, MongoDB, or LDAP. The pillar modules page in the documentation lists all supported sources.

In this post, I will provide a specific explanation of how to retrieve pillar data from a MongoDB server. Although this configuration is explained into the salt.pillar.mongo documentation section, I will provide step-by-step explanations on how to install the required packages on a Debian system, insert data in MongoDB, and configure Salt to make the all thing works, avoiding some pitfalls.

Pre-requisites and objectives

You should have a working Salt setup with a functional minion connected to a master.

The following instructions assume they run Debian.

The Salt minion ID in this example is minion1.example.com.

The objective is to add a data tree coming from MongoDB under the pillar key mongo_pillar, retrievable using this command on the master :

salt minion1.example.com pillar.item mongo_pillar

Technical steps

Install MongoDB and the MongoDB Python module

apt-get install mongodb python-pymongo python-pymongo-ext

Be sure to have a valid installation of the Python MongoDB module, otherwise loading of the Pillar MongoDB module will fail silently. I've opened a pull request to document this dependency in the Salt documentation.

Connect to MongoDB

# mongo
MongoDB shell version: 2.0.6
connecting to: test
>

For more informations about the MongoDB client, see the official MongoDB documentation, or these notes in my MongoDB knowledge page.

Create a MongoDB database and collection to store pillar data

Using the MongoDB client, create a new database called pillar, whose name will be later referenced in Salt configuration :

use pillar

Insert this example pillar data tree in the database :

db.pillar.insert({
    _id: 'minion1.example.com',
    mongo_pillar: {
        key1: "value1",
        key2: "value2",
    }})

The _id field matches the name of the minion to whom the data will be presented.

The other fields (here mongo_pillar) contain the desired pillar data which will be anchored at the root of the pillar tree.

One drawback of external pillars : you cannot use globbing to export the same set of data to several minions, like you can do using the Pillar topfile (/srv/pillar/top.sls).

Configure Salt to access MongoDB

Edit /etc/salt/master and add these lines :

mongo.db: "pillar"
mongo.host: "localhost"
ext_pillar:
    - mongo: {}

Note this is also possible to specify a non-standard port (mongo.port) and authentication credentials (mongo.user and mongo.password).

The ext_pillar parameter specify a list of external Pillar interfaces. Here we only use the mongo interface with an empty parameters dictionary ({}).

Available parameters allow to specify a different collection name or ID field, filter which fields should be exported in the Pillar tree, or perform string transformations on the ID field.

Test

Restart the master, then execute the following command to retrieve our data :

salt minion1.example.com pillar.item mongo_pillar

In case of failure, check master logs (/var/log/salt/master/) and MongoDB logs (/var/log/mongodb/).

Cool, you've now a working Salt/Pillar/MongoDB setup !