developerstoolbox / ini-file-parser Goto Github PK

A bash implementation of an ini (config) file parser.

License: MIT License

Shell 100.00%

ini-file-parser's Introduction

Overview

Bash does not come with a way to process ini/config files, this script is an attempt at creating a generic easy to use solution to this problem and for no other reason than because I could.

How it works

The script works by reading in a normal ini file and creates 2 arrays per section. One for the keys and one for the values. The reason of this is that you cannot declare an associative array globally from within a loop which is within a function (in bash 4.3 at least) so this was the best work around that was available.

Example ini file

There are 2 config files provided as examples.

Name	Description
simple example	Simple config file, demonstrating sections and key=value key pairs.
complete example	Complex config file, demonstrating all of the processing rules, warnings and error conditions.

Simple config file

[section1]
value1=abcd
value2=1234

[section2]
value1=1234
value2=5678

[section3]
value1=abcd
value2=1234
value3=a1b2
value_4=c3d4

Example usage

There is a complete working example available, parse-example.sh, but a 'snippet' example is given below, to show a simple single value extraction.

#!/usr/bin/env bash

# Load in the ini file parser
source ini-file-parser.sh

# Load and process the ini/config file
process_ini_file 'example.conf'

# Display a specific value from a specific section
echo $(get_value 'section1' 'value1')

# Display the same value as above but using the global variables created as part of the processing.
echo $section1_value1

Processing rules

There are a number of rules / assumptions that have been coded in that might give you a result different to the one you might expect:

Empty lines are ignored.
Lines starting with comments (# or ;) are ignored.
The 'default' section is called 'default' - This is for any name=value pair defined before the first section.
Section names will only contains alpha-numeric characters and underscores - everything else is removed.
Section names are case sensitive (by default).
Key names have leading and trailing spaces removed.
Key names replace all punctuation and blank characters (space, tab etc) with underscore and squish (remove multiple underscores).
Value fields have in-line comments removed.
Value fields have leading and trailing spaces removed.
Value fields have leading and trailing quotes removed.

In addition to the data arrays that are created a set of variables are also created to allow for simple direction access to values, these are in the format: section_key=value.

It is worth noting that the global variables are created AFTER the processing rules have been applied, so the section name and the key name might differ from what you expect.

Subroutines

There are currently 4 subroutines that are exposed.

process_ini_file - Load a names ini/config file and create the required arrays.
get_value - Request a specific value (by key name) from a specific section.
display_config - Display the processed (clean) ini/config in ini/config file format.
display_config_by_section - As above but for a specific named section.

Global overrides

These variables allow us to override the parse script defaults, they can be used by the calling script, but they must be set BEFORE the ini/config file is processed.

Name	Description	Default	Accepted Values
case_sensitive_sections	Should section names be case sensitive?	true	true / false
case_sensitive_keys	Should key names be case sensitive?	true	true / false
show_config_warnings	Should we show config warnings?	true	true / false
show_config_errors	Should we show config errors?	true	true / false

Example override usage

#!/usr/bin/env bash

# Turn off config warnings
show_config_warnings=false

# Make Section names case-insensitive
case_sensitive_sections=false

# Load in the ini file parser
source ini-file-parser.sh

# Load and process the ini/config file
process_ini_file 'example.conf'

# Display a specific value from a specific section
echo $(get_value 'section1' 'value1')

# Display a value using the global variable created as part of the processing.
echo $section1_value1

ini-file-parser's People

Contributors

Stargazers

Watchers

Forkers

jmatias zhouyu0615 shakahl johanchane brucejin cindyhu-gd mrosama printline512

ini-file-parser's Issues

[Bug]:

What happened?

I create an ini file with a comment that looks like a key/value pair where value is missing:

cat <<EOT > test.ini
[section]
; x = 
y = 2
EOT

I source the script and load the ini file:

source ini-file-parser.sh
process_ini_file test.ini

This results in an error message:

[ ERROR ] line 2: No value

Expected behavior

I expect the script to ignore line 2 because it is a comment.

How do we reproduct the bug?

Open an terminal.
Change the terminal's working directory to …/ini-file-parser/src.
Copy the above script and paste it into the terminal.

Relevant log output

[ ERROR ] line 2: No value

Screeenshots

No response

Additional information

No response

Code of Conduct

I agree to follow this project's Code of Conduct

[Bug]: File suffix other than .ini gives warnings

What happened?

I am executing the following:

cat <<EOT > test.ini
[my_test]
test_1 = A
EOT

cp test.ini test.conf

source ini-file-parser.sh

echo "test.ini"
process_ini_file test.ini

echo "test.conf"
process_ini_file test.conf

The resulting output:

test.ini
test.conf
[ WARNING ] key test_1 - Defined multiple times within section my_test

Expected behavior

The expected output:

test.ini
test.conf

How do we reproduct the bug?

Open an terminal.
Change the terminal's working directory to …/ini-file-parser/src.
Copy the above script and paste it into the terminal.

Relevant log output

[ WARNING ] key test_1 - Defined multiple times within section my_test

Screeenshots

No response

Additional information

No response

Code of Conduct

I agree to follow this project's Code of Conduct

[Bug]: substring mismatch in section name?

What happened?

So, I gave it a spin against systemd-resolved and its INI-styled config file:

And I think the section name did not terminate the search or
there is at keyword-level pattern matching a minor substring inadvert match ('DNS' matched against 'FallbackDNS' as positive thereby pulling in the value from 'FallbackDNS' from another section!

Tweaked code whose filename is ini_find_keyword.sh:

#!/bin/bash
# source: https://github.com/DevelopersToolbox/ini-file-parser/blob/master/src/ini-file-parser.sh
# Never reinvent the wheel, this is the best INI parser found in bash language.
# But, an empty value is an error; we tweaked it for an info
# -------------------------------------------------------------------------------- #
# Description                                                                      #
# -------------------------------------------------------------------------------- #
# A 'complete' ini file parsers written in pure bash (4), it was written for no    #
# other reason that one did not exist. It is completely pointless apart from some  #
# clever tricks.                                                                   #
# -------------------------------------------------------------------------------- #

# -------------------------------------------------------------------------------- #
# Global Variables                                                                 #
# -------------------------------------------------------------------------------- #
# Global variables which can be set by the calling script, but need to be declared #
# here also to ensure the script is clean and error free.                          #
#                                                                                  #
# case_sensitive_sections - should section names be case sensitive                 #
# case_sensitive_keys     - should key names be case sensitive                     #
# show_config_warnings    - should we show config warnings                         #
# show_config_errors      - should we show config errors                           #
# -------------------------------------------------------------------------------- #

declare case_sensitive_sections
declare case_sensitive_keys
declare show_config_warnings
declare show_config_errors

# -------------------------------------------------------------------------------- #
# Default Section                                                                  #
# -------------------------------------------------------------------------------- #
# Any values that are found outside of a defined section need to be put somewhere  #
# so they can be recalled as needed. Sections is set up with a 'default' for this  #
# purpose.                                                                         #
# -------------------------------------------------------------------------------- #

DEFAULT_SECTION='default'

sections=( "${DEFAULT_SECTION}" )

# -------------------------------------------------------------------------------- #
# Local Variables                                                                  #
# -------------------------------------------------------------------------------- #
# The local variables which can be overridden by the global variables above.       #
#                                                                                  #
# local_case_sensitive_sections - should section names be case sensitive           #
# local_case_sensitive_keys     - should key names be case sensitive               #
# local_show_config_warnings    - should we show config warnings                   #
# local_show_config_errors      - should we show config errors                     #
# -------------------------------------------------------------------------------- #

local_case_sensitive_sections=true
local_case_sensitive_keys=true
local_show_config_warnings=true
local_show_config_errors=true

# -------------------------------------------------------------------------------- #
# Set Global Variables                                                             #
# -------------------------------------------------------------------------------- #
# Check to see if the global overrides are set and if so, override the defaults.   #
#                                                                                  #
# Error checking is in place to ensure that the override contains a valid value of #
# true or false, anything else is ignored.
# -------------------------------------------------------------------------------- #

function setup_global_variables
{
    if [[ -n "${case_sensitive_sections}" ]] && [[ "${case_sensitive_sections}" = false || "${case_sensitive_sections}" = true ]]; then
         local_case_sensitive_sections=$case_sensitive_sections
    fi

    if [[ -n "${case_sensitive_keys}" ]] && [[ "${case_sensitive_keys}" = false || "${case_sensitive_keys}" = true ]]; then
         local_case_sensitive_keys=$case_sensitive_keys
    fi

    if [[ -n "${show_config_warnings}" ]] && [[ "${show_config_warnings}" = false || "${show_config_warnings}" = true ]]; then
         local_show_config_warnings=$show_config_warnings
    fi

    if [[ -n "${show_config_errors}" ]] && [[ "${show_config_errors}" = false || "${show_config_errors}" = true ]]; then
         local_show_config_errors=$show_config_errors
    fi
}

# -------------------------------------------------------------------------------- #
# in Array                                                                         #
# -------------------------------------------------------------------------------- #
# A function to check to see if a given value exists in a given array.             #
# -------------------------------------------------------------------------------- #

function in_array()
{
    local haystack="${1}[@]"
    local needle=${2}

    for i in ${!haystack}; do
        if [[ ${i} == "${needle}" ]]; then
            return 0
        fi
    done
    return 1
}

# -------------------------------------------------------------------------------- #
# Show Warning                                                                     #
# -------------------------------------------------------------------------------- #
# A wrapper to display any configuration warnings, taking into account if the      #
# local_show_config_warnings flag is set to true.                                  #
# -------------------------------------------------------------------------------- #

function show_warning()
{
    if [[ "${local_show_config_warnings}" = true ]]; then
        format=$1
        shift;

        # shellcheck disable=SC2059
        printf "[ WARNING ] ${format}" "$@";
    fi
}

# -------------------------------------------------------------------------------- #
# Show Error                                                                       #
# -------------------------------------------------------------------------------- #
# A wrapper to display any configuration errors, taking into account if the        #
# local_show_config_errorss flag is set to true.                                   #
# -------------------------------------------------------------------------------- #

function show_error()
{
    if [[ "${local_show_config_errors}" = true ]]; then
        format=$1
        shift;

        # shellcheck disable=SC2059
        printf "[ ERROR ] ${format}" "$@";
    fi
}

# -------------------------------------------------------------------------------- #
# Process Section Name                                                             #
# -------------------------------------------------------------------------------- #
# Once we have located a section name within the given config file, we need to     #
# 'cleanse' the value.                                                             #
# -------------------------------------------------------------------------------- #

function process_section_name()
{
    local section=$1

    section="${section##*( )}"                                                     # Remove leading spaces
    section="${section%%*( )}"                                                     # Remove trailing spaces
    section=$(echo -e "${section}" | tr -s '[:punct:] [:blank:]' '_')              # Replace all :punct: and :blank: with underscore and squish
    section=$(echo -e "${section}" | sed 's/[^a-zA-Z0-9_]//g')                     # Remove non-alphanumberics (except underscore)

    if [[ "${local_case_sensitive_sections}" = false ]]; then
        section=$(echo -e "${section}" | tr '[:upper:]' '[:lower:]')               # Lowercase the section name
    fi
    echo "${section}"
}

# -------------------------------------------------------------------------------- #
# Process Key Name                                                                 #
# -------------------------------------------------------------------------------- #
# Once we have located a key name on a given line, we need to 'cleanse' the value. #
# -------------------------------------------------------------------------------- #

function process_key_name()
{
    local key=$1

    key="${key##*( )}"                                                             # Remove leading spaces
    key="${key%%*( )}"                                                             # Remove trailing spaces
    key=$(echo -e "${key}" | tr -s '[:punct:] [:blank:]' '_')                      # Replace all :punct: and :blank: with underscore and squish
    key=$(echo -e "${key}" | sed 's/[^a-zA-Z0-9_]//g')                             # Remove non-alphanumberics (except underscore)

    if [[ "${local_case_sensitive_keys}" = false ]]; then
        key=$(echo -e "${key}" | tr '[:upper:]' '[:lower:]')                       # Lowercase the section name
    fi
    echo "${key}"
}

# -------------------------------------------------------------------------------- #
# Process Value                                                                    #
# -------------------------------------------------------------------------------- #
# Once we have located a value attached to a key, we need to 'cleanse' the value.  #
# -------------------------------------------------------------------------------- #

function process_value()
{
    local value=$1

    value="${value%%\;*}"                                                          # Remove in line right comments
    value="${value%%\#*}"                                                          # Remove in line right comments
    value="${value##*( )}"                                                         # Remove leading spaces
    value="${value%%*( )}"                                                         # Remove trailing spaces

    value=$(escape_string "$value")

    echo "${value}"
}

# -------------------------------------------------------------------------------- #
# Escape string                                                                    #
# -------------------------------------------------------------------------------- #
# Replace ' with SINGLE_QUOTE to avoid issues with eval.                           #
# -------------------------------------------------------------------------------- #

function escape_string()
{
    local clean

    clean=${1//\'/SINGLE_QUOTE}
    echo "${clean}"
}

# -------------------------------------------------------------------------------- #
# Un-Escape string                                                                 #
# -------------------------------------------------------------------------------- #
# Convert SINGLE_QUOTE back to ' when returning the value to the caller.           #
# -------------------------------------------------------------------------------- #

function unescape_string()
{
    local orig

    orig=${1//SINGLE_QUOTE/\'}
    echo "${orig}"
}

# -------------------------------------------------------------------------------- #
# Parse ini file                                                                   #
# -------------------------------------------------------------------------------- #
# Read a named file line by line and process as required.                          #
# -------------------------------------------------------------------------------- #

function process_ini_file()
{
    local line_number=0
    local section="${DEFAULT_SECTION}"
    local key_array_name=''

    setup_global_variables

    shopt -s extglob

    while read -r line; do
        line_number=$((line_number+1))

        if [[ $line =~ ^# || -z $line ]]; then                                 # Ignore comments / empty lines
            continue;
        fi

        if [[ $line =~ ^"["(.+)"]"$ ]]; then                                   # Match pattern for a 'section'
            section=$(process_section_name "${BASH_REMATCH[1]}")

            if ! in_array sections "${section}"; then
                eval "${section}_keys=()"                                      # Use eval to declare the keys array
                eval "${section}_values=()"                                    # Use eval to declare the values array
                sections+=("${section}")                                       # Add the section name to the list
            fi
        elif [[ $line =~ ^(.*)"="(.*) ]]; then                                 # Match patter for a key=value pair
            key=$(process_key_name "${BASH_REMATCH[1]}")
            value=$(process_value "${BASH_REMATCH[2]}")

            if [[ -z ${key} ]]; then
                show_error 'line %d: No key name\n' "${line_number}"
            #elif [[ -z ${value} ]]; then
                #show_warning 'line %d: No value\n' "${line_number}"
            else
                if [[ "${section}" == "${DEFAULT_SECTION}" ]]; then
                    show_warning '%s=%s - Defined on line %s before first section - added to "%s" group\n' "${key}" "${value}" "${line_number}" "${DEFAULT_SECTION}"
                fi

                eval key_array_name="${section}_keys"

                if in_array "${key_array_name}" "${key}"; then
                    show_warning 'key %s - Defined multiple times within section %s\n' "${key}" "${section}"
                fi
                eval "${section}_keys+=(${key})"                               # Use eval to add to the keys array
                eval "${section}_values+=("${value}")"                         # Use eval to add to the values array
                eval "${section}_${key}='${value}'"                            # Use eval to declare a variable
            fi
        fi
    done < "$1"
}

# -------------------------------------------------------------------------------- #
# Get Value                                                                        #
# -------------------------------------------------------------------------------- #
# Retrieve a value for a specific key from a named section.                        #
# -------------------------------------------------------------------------------- #

function get_value()
{
    local section=''
    local key=''
    local value=''
    local keys=''
    local values=''

    section=$(process_section_name "${1}")
    key=$(process_key_name "${2}")

    eval "keys=( \"\${${section}_keys[@]}\" )"
    eval "values=( \"\${${section}_values[@]}\" )"

    for i in "${!keys[@]}"; do
        if [[ "${keys[$i]}" = "${key}" ]]; then
            orig=$(unescape_string "${values[$i]}")
            printf '%s' "${orig}"
        fi
    done
}

# -------------------------------------------------------------------------------- #
# Display Config                                                                   #
# -------------------------------------------------------------------------------- #
# Display all of the post processed configuration.                                 #
#                                                                                  #
# NOTE: This is without comments etec.                                             #
# -------------------------------------------------------------------------------- #

function display_config()
{
    local section=''
    local key=''
    local value=''

    for s in "${!sections[@]}"; do
        section=${sections[$s]}

        printf '[%s]\n' "${section}"

        eval "keys=( \"\${${section}_keys[@]}\" )"
        eval "values=( \"\${${section}_values[@]}\" )"

        for i in "${!keys[@]}"; do
            orig=$(unescape_string "${values[$i]}")
            printf '%s=%s\n' "${keys[$i]}" "${orig}"
        done
    printf '\n'
    done
}

# -------------------------------------------------------------------------------- #
# Display Config by Section                                                        #
# -------------------------------------------------------------------------------- #
# Display all of the post processed configuration for a given section.             #
#                                                                                  #
# NOTE: This is without comments etec.                                             #
# -------------------------------------------------------------------------------- #

function display_config_by_section()
{
    local section=$1
    local key=''
    local value=''
    local keys=''
    local values=''

    printf '[%s]\n' "${section}"

    eval "keys=( \"\${${section}_keys[@]}\" )"
    eval "values=( \"\${${section}_values[@]}\" )"

    for i in "${!keys[@]}"; do
        orig=$(unescape_string "${values[$i]}")
        printf '%s=%s\n' "${keys[$i]}" "${orig}"
    done
    printf '\n'
}

# -------------------------------------------------------------------------------- #
# End of Script                                                                    #
# -------------------------------------------------------------------------------- #
# This is the end - nothing more to see here.                                      #
# -------------------------------------------------------------------------------- #

if [ -n "$UNITTEST" ]; then
  # temp_file="$(mktemp "/tmp/unittest_$(basename ${0})-XXXX.tmp")"
  temp_file="/tmp/unittest.ini-file-parser"
  cat << TEST_EOF | tee "$temp_file"
#
varBeforeDefault=1
more_var_before_default=2
# DNS=1.1.1.1

[Resolve]
DNS=    # inline comment
#DNS=2.2.2.2
FallbackDNS=8.8.8.8    # inline comment2

[Notaresolve]
DNS=8.8.8.8
FallbackDNS=

TEST_EOF
  # source ini-file-parser.sh
  process_ini_file "$temp_file"
  echo "Result-DNS is: $(get_value 'Resolve' 'DNS')"
  echo "default varBeforeDefault is: $default_varBeforeDefault"
  echo "default DNS is: $default_DNS"  # should be commented out
  echo "Resolve DNS is: $(get_value 'Resolve' 'DNS') # should be empty"
  echo "Resolve DNS is: $Resolve_DNS # should be empty"
  echo "'Notaresolve' DNS is: $(get_value 'Notaresolve' 'DNS') # should be 8.8.8.8"
  echo "Resolve FallbackDNS is: $(get_value 'Resolve' 'FallbackDNS') # should be 8.8.8.8"
fi

Expected behavior

keyword search, given a section name, should have terminated its search when encountering a "different" section name.

However, I think this is the case where [Resolve] section got properly found using the section-based pattern Resolve and also found the same section-based pattern in [Notaresolve] section being "mis-sub-"matched as in resolve. My guess here.

Offtopic: (not the bug here but a potential one is to continue search pattern for keyword if the same section name "RE"-appears.)

How do we reproduct the bug?

To see unit test against the slightly modified snippet of code, execute the following:

$ UNITTEST=1 ./ini_find_keyword.sh

Relevant log output

No response

Screeenshots

No response

Additional information

Bash version: GNU bash, version 5.1.4(1)-release (x86_64-pc-linux-gnu)
Distro: Debian 11 (bullseye) Linux
OS: 5.10.0-11-amd64 #1 SMP Debian 5.10.92-1 (2022-01-18) x86_64 GNU/Linux

Code of Conduct

I agree to follow this project's Code of Conduct

[Bug]: Declare the variable with eval fails if value contains space char

What happened?

Declare the variable with eval (line 256) fails if the value contains a space char (ascii 0x20).

For example, the following ini file:
[section1]
value1=abc def

Results to the following error:
ini-file-parser.sh: line 256: def: command not found

Expected behavior

Values containing spaces should work.

How do we reproduct the bug?

Parse a ini file with a value that contains a space char

Relevant log output

No response

Screeenshots

No response

Additional information

In line 256, enclose the value varaible in double quotes and it works:

eval "${section}_${key}=\"${value}\""

Code of Conduct

I agree to follow this project's Code of Conduct

Recommend Projects

React

A declarative, efficient, and flexible JavaScript library for building user interfaces.
Vue.js

🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.
Typescript

TypeScript is a superset of JavaScript that compiles to clean JavaScript output.
TensorFlow

An Open Source Machine Learning Framework for Everyone
Django

The Web framework for perfectionists with deadlines.
Laravel

A PHP framework for web artisans
D3

Bring data to life with SVG, Canvas and HTML. 📊📈🎉

Recommend Topics

javascript

JavaScript (JS) is a lightweight interpreted programming language with first-class functions.
web

Some thing interesting about web. New door for the world.
server

A server is a program made to process requests and deliver data to clients.
Machine learning

Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.
Visualization

Some thing interesting about visualization, use data art
Game

Some thing interesting about game, make everyone happy.

Recommend Org

Facebook

We are working to build community through open source technology. NB: members must have two-factor auth.
Microsoft

Open source projects and samples from Microsoft.
Google

Google ❤️ Open Source for everyone.
Alibaba

Alibaba Open Source for everyone
D3

Data-Driven Documents codes.
Tencent

China tencent open source team.