preloader

Tips and Tricks

Table of Contents

Overview

An unsorted list of tips and tricks for SaltStack.

Execution Modules and Python

A salt execution module is just a python function.

e.g. execution module test.ping is the ping function inside the test module

This makes it easy to go to the SaltStack repo and find the function you are looking for.

Salt-Open Repo

Salt CLI

Gathering information

  • List all available modules on a minion target
salt '*' sys.list_modules
  • List the documentation for a module
salt '*' sys.doc MODULENAME
  • List all the arguments available for an execution module
salt '*' sys.argspec test.ping
  • List the results in groups based on differential
salt-run survey.diff '*' pkg.version kernel
salt-call sys.runner_doc survey.diff

Salt Utilities

  • Show all the Jobs that are running
salt '*' saltutil.running
  • Terminate all running jobs on a minion
salt '*' saltutil.term_all_jobs
  • Kill all running jobs on a minion
salt '*' saltutil.kill_all_jobs
  • Restart the Minion from the master
salt '*' cmd.run_bg 'salt-call --local service.restart salt-minion'

Top file

  • Run a highstate
# New
salt '*' state.apply

# Legacy
salt '*' state.highstate
  • See what the top file matches are for the environment
salt '*' state.show_top

File Server

  • List all the files being served by Salt
salt '*' cp.list_master
  • List all the directories being served by Salt
salt '*' cp.list_master_dirs
  • List all of the available state files in an environment
salt '*' cp.list_states
  • List all of the files cached on the minion
salt '*' cp.list_minion
  • Copy a file from the master to a Minion
salt '*' cp.get_file salt://files/metadata.py "/usr/lib/python2.7/site-packages/salt/grains/metadata.py"

Grains

  • List all available grains (key only)
salt '*' grains.ls
  • List all available grains
salt '*' grains.items
  • List a specific grain
salt '*' grains.item $NAME
  • List multiple specific grains
salt '*' grains.item $NAME1 $NAME2

Pillars

  • Refresh Pillar data to the minions
salt '*' saltutil.pillar_refresh
  • List all pillars (key only)
salt '*' pillar.ls
  • List all pillars (re-generate)
salt '*' pillar.items
  • List all pillars (from memory)
salt '*' pillar.raw
  • List a specific pillar (from memory)
salt '*' pillar.item  $PILLAR
  • List a specific nested pillar
salt '*' pillar.item $LEVEL1:$LEVEL2:$PILLAR
  • Render the pillar top.sls
salt '*' state.show_top

Network

  • Verify network access from a minion to another location
salt '*' network.connect somedomain.com 443
  • List all the network address for a minion
salt '*' network.ipaddrs
  • Show all active TCP connections on a minion
salt '*' network.active_tcp
  • Perform a DNS dig on a minion
salt '*' network.dig somedomain.com
  • Perform a HTTP/S URL and get a return (curl)
salt '*' http.query http://localhost/health-check
  • Wait for a successful HTTP/S query before continuing
# NOTE: handy to verify that a service is up before returning to service.
salt '*' http.wait_for_successful_query http://localhost/health-check wait_for=60

Services

  • Get the status of a service
salt '*' service.status SERVICE-NAME
  • Reload a service
salt '*' service.reload SERVICE-NAME
  • Restart a service
salt '*' service.restart SERVICE-NAME
  • Get all services
salt '*' service.get_all

Files

  • Grep a file (shows content)
salt '*' file.grep /path/file patter
  • Sed a file (replaces content)
salt '*' file.replace /path/file pattern="string" repl="#string" append_if_not_found=True

salt '*' file.replace /etc/salt/minion pattern='cloud_ec2_grains.*$' repl='cloud_ec2_grains: True' append_if_not_found=True
  • Search a file (Return True/false)
salt '*' file.search /path/file pattern
  • Show how much space a file is using
salt '*' file.diskusage /path/file
  • Find a file
salt '*' file.find '/usr' iname=\*.log

Command Module

  • Run a command
salt '*' cmd.run 'echo "This is an example"'
  • Execute some code
salt '*' cmd.exec python3 'print("Hello World!")'
  • Does a minion have an executable name
salt '*' cmd.has_exec EXECUTABLE-NAME
  • Which
salt '*' cmd.which EXECUTABLE-NAME
  • Run a script on a Minion
salt '*' cmd.script salt://scripts/my_script.sh 'argument1"

Jinja

  • Test a jinja template by filling in variables (On the Master)
salt-run salt.cmd slsutil.renderer /srv/salt/top.sls
  • Test a jinja template by filling in variables (On a Minion, from the Master)
salt '*' slsutil.renderer salt://top.sls 'jinja'
  • Parse a jinja template and store it on the minion for review
salt '*' cp.get_template salt://top.sls /tmp/salt.sls template=jinja

Peer Publishing

Salt can allow a Minion to run a command on another Minion

  • Example: Allow a Web server to run the haproxy module on a Proxy
  # Master configuration
  peer:
    'web*':
      'haproxy*':
          - haproxy.*
  • Example publish command
salt web1 publish.publish haproxy1 haproxy.disable_server arg="['id', 'backend']"

Subset

  • Salt can only show a return from a Subset of the matched minions
salt --subset=1 -G 'roles:web' cmd.run 'cat /etc/nginx'

Batchsize

  • Salt execute commands one host at a time
salt -b1 -G 'roles:web' service.restart haproxy

Fail Hard

  • Stop processing if a state fails (handy for batch operations)
salt --failhard -b1 -G 'roles:web' service.restart nginx

Blackout Mode

Salt supports minion blackouts. During a blackout, the minion will reject all commands except saltutil.refresh_pillar

  • Set the following pillar value
minion_blackout: True
  • You can also whitelist functions that will still be allowed during a blackout.
minion_blackout_whitelist:
  - test.ping
  - pillar.get

Using Salt in Shell scripts

  • Example: Restart a service on a clustered server and wait for something to be true
# Get a list of servers
SERVERS=($(salt -G 'roles:app' test.ping --output=text | sort | grep -Po '.*(?=: True)'))

for server in ${SERVERS[@]};
do

    salt $server service.restart DaonIdentityXSRP
    while [ ! salt $server cmd.run "curl 127.0.0.1:XXXX/Health_Check" ];
    then

        sleep 300

    done

done
  • Example: Recycle Bad Minions
BADMINIONS=($(salt -G 'roles:proxy' test.ping --output=text | grep -Po '.*(?=: Minion did not return)'))

for minion in ${BADMINIONS[@]};
do

    salt $minion service.restart salt-minion;

done

Salt State file structure migrations

image 
# Set the ServerRealm

{% set structure = salt['grains.get']}('structure', 'v1') %}

{% if structure == 'v1' %}

    ...

{% endif %}

{% if structure == 'v2' %}

    {% set role = salt['grains.get']('role', 'None') %}
    'G@role:{{ role }}':
      - match: compound
      - 'role.{{ role }}'

{% endif %}