1. Introduction
Almost every application needs configuration data to work as intended. This configuration data normally comes in form of property files, json- or yaml-formatted files, as environment variables set by a start-script or in some other form.
The OS Connector is a ConfigSeeder component that provides configuration data in form of files to an OS (Linux or Windows).
Two types of files can be provided to applications:
-
Files with static Content
-
Files with templating (e.g. content of file can differ depending on filter critera)
1.1. Use cases
-
Provide Configuration Data at installation time
During the install process, the OS Connector is executed and provides configuration files. These files are used by the installation process and copied to the right loation. At startup time, the OS Connector (and therefore the ConfigSeeder) isn’t required any more.
You can also execute the OS Connector with a tool like Ansible to setup a Linux VM (There will also be the possibility to retrieve configuration data directly with Ansible with an Ansible Plugin. Please contact us if you are interested in trying out our Ansible Integration).
-
Provide Configuration Data at startup time
Before an application is started, the OS Connector is executed and provides configuration data for the application.
The OS Connector is also available as docker container (
configseeder/linux-connector:<version>
) which can be used as init-container.
2. Quickstart
2.1. Introduction
2.1.1. Content of the quickstart manual
The OS Connector is able to provide configuration files to a linux or windows machine. This quickstart guide gives an introduction to the use of the OS Connector.
You will understand, how
-
static configuration files are defined in ConfigSeeder
-
configuration files with templating are defined in ConfigSeeder
-
the OS Connector provides configuration files to Linux or Windows
This quickstart guide also offers links to more detailed information.
Be advised that the OS Connector is a ConfigSeeder module under heavy development and currently available as a tech preview.. The structure of the Configuration Values / Assemblies used to configure files can be changed in the future. |
2.1.2. Preconditions
-
ConfigSeeder up and running
-
Connectivity from the Linxu/Windows VM running the OS Connector to the ConfigSeeder
-
Licence which supports the use of OS Connector
2.2. Prepare configuration groups in ConfigSeeder
-
Create a Configuration Group
os-connector-quickstart
(the name of the Configuration Group can be chosen as desired)When creating the Configuration Group, just enable the filter columns Environment and Context and leave the security options on their default values.
-
Create the two environments
TEST
andPROD
(the name of the Environments can be chosen as desired) -
Create two API Keys with permission to access the Configuration Group
os-connector-quickstart
One API Key should have permissions to access the Test-environment, the other one for Production.Copy the generated API Key to a secure location. ConfigSeeder doesn’t store API Keys so you can’t retrieve it again.
2.3. Scenario 1: Static configuration
2.3.1. Prepare configuration values in ConfigSeeder
-
Open the Configuration Group
os-connector-quickstart
, switch to the Value Assemblies-Tab and create the following Assembly of TypeFile: static
:Remarks:
-
In the example
meta.filename
is set toquickstart-static.txt
. It can also be set to a absolute or relative path. -
With the value shown in the screenshot, the file will be created at the location where the OS Connector is executed.
-
If you are trying out the OS Connector on Linux, pathes can be entered like
<path>/<filename>
where<path>
can be absolute or relative -
If you are trying out the OS Connector on Windows, be sure to enter a valid windows path like
c:\temp...
-
2.3.2. Setup OS Connector
-
Download und unpack the OS Connector to your system (see https://preview.configseeder.com/download/, depending on your os please use
linux-connector
orwindows-connector
) -
Create the file
connector.yaml
with the follwing content:connector: configSeeder: configurationGroupKeys: "os-connector-quickstart" tenantKey: "configseeder" (1) environmentKey: "TEST" (2) context: "example-static" (3) serverUrl: "http://url" (4) connectionTimeout: 2000 readTimeout: 2000 apiKey: "eyJraWQiO..." (5)
1 Tenant, in which the configuration is stored. The name of the default tenant is default
(it is only required to specify the tenant if not the default tenant is used. If no tenant is specifieddefault
is used by convention)2 Environment, for which the configuration should be read. 3 The context, for which the configuration should be retrieved. Is normally the name of the os (hostname) but can be any value or not used at all. If multiple os connectors are used, the combination of environment and context should be unique. In this quickstart guide, context is used to distinguish the different scenarios. 4 URL under which the ConfigSeeder is reachable. 5 Api Key with permissions to access the configured configuration groups and enviroment.
2.3.3. Execute os-connector
Start the OS Connector by executing ./linux-connector
or executing windows-connector.exe
As a result, the OS Connector should log Created file /tmp/quickstart-static.txt for Application quickstart
and the
file /tmp/quckstart-static.txt
should have been created with the configured content.
If the executable is run again, the OS Connector should log something like LastUpdate matches, no action required for
file /tmp/quickstart-static.txt for Application quickstart
and the file should be unchanged. If the content is
changed and the command rerun, the OS Connector will change content of the file so it matches the configured content
again.
2.4. Scenario 2: Configuration with templating
2.4.1. Prepare configuration values in ConfigSeeder
-
Open the Configuration Group
os-connector-quickstart
, switch to the Value Assemblies-Tab and create the following Assembly of TypeFile: template
:Remarks:
-
In this example there is no environment set. The content of the file is controlled by the values which are used for the templating, see next step.
-
-
Open the Configuration Group
os-connector-quickstart-values
and create the following Configuration Values:
2.4.2. Setup OS Connector
-
Ensure that the OS Connector is available.
-
Change the
connector.yaml
toconnector: configSeeder: configurationGroupKeys: "os-connector-quickstart" tenantKey: "default" environmentKey: "TEST" context: "example-templating" (1) serverUrl: "http://url" connectionTimeout: 2000 readTimeout: 2000 apiKey: "eyJraWQiO..."
1 change context to example-templating
2.4.3. Execute OS Connector
Start the OS Connector by executing ./linux-connector
or executing windows-connector.exe
As a result, the OS Connector should log Created file quickstart-templating.txt for Application quickstart
and
the file quckstart-template.txt
should have been created with the content configured for test.
3. Configuration Options
3.1. Ways to configure the Connector
This section describes every available configuration option available for OS Connector and it’s modules.
There are three ways to configure the connector:
-
Configuration by file (lowest priority)
Default filename is
connector.yaml
, the connector looks for the file in the directoy it is started from. The filename can be overwritten, see Configuration Options.Example:
connector: log: level: DEBUG file: directoryWhitelist: - /tmp configSeeder: tenantKey: "configseeder" environmentKey: "TEST" configurationGroupKeys: "os-connector-integrationtest" serverUrl: "https://staging-postgres-config-seeder.oneo.cloud"
-
Configuration by environment variables
Example:
export CONNECTOR_LOG_LEVEL=DEBUG export CONNECTOR_CONFIGSEEDER_APIKEY=xxxx ... linux-connector
-
Configuration by startup parameter (highest priority)
Example:
linux-connector -connector.log.level=DEBUG -connector.configseeder.apikey=xxx ...
It is possible to mix the different ways to configure the connector. You probably want to configure the connector by
file and provide the API Key with an environment variable so it isn’t readable by anyone (You don’t want to provide the
API Key as a startup parameter because the values of startup parameters can be retrieved by anyone using ps awx
).
Priorities of ways to configure the connector
|
3.2. Configuration Options
The OS Connector can be configured by the following options:
All keys are prefixed with |
Key | Mandatory | Value |
---|---|---|
|
no |
Location and name of the configfile containing the connector configuration. Defaults to |
|
yes |
API Key used by Connector to access ConfigSeeder. The API Key must be of Type |
|
yes |
Defines which configuration groups should be read by the Connector. |
|
no |
Optional for filtering configuration values by |
|
no |
Connection Timeout, defaults to |
|
no |
Optional for filtering configuration values by Defaults to now. |
|
yes |
Environment for which the Connector is running. |
|
no |
Defines, how many times the connector should try to connect to ConfigSeeder Defaults to |
|
no |
Read Timeout, defaults to |
|
no |
Time in Milliseconds between failed tries to connect to ConfigSeeder. Defaults to |
|
yes |
URL under which ConfigSeeder can be reached. |
|
no |
Defaults to Tenant |
|
no |
Optional for filtering configuration values by |
|
no |
Set to Defaults to |
|
no |
If set, the OS Connector is only allowed to manage files in the specified directories. |
|
no |
Set to |
|
no |
Default file mode, if no mode is specified. Defaults to |
|
no |
Optional prefix for files with a relative path. If set, all relative files are prefixed with this value. |
|
no |
Postfix for keys which will be replaced by ConfigSeeder. Defaults to |
|
no |
Prefix for keys which will be replaced by ConfigSeeder. Defaults to |
|
no |
Name of the Logfile if logging to a file is configured. Defaults to |
|
no |
ogLevel for the connector, valid levels are: trace, debug, info, warning, error, fatal, panic. Defaults to 'info' |
|
no |
Target for the connector logging, valid targets are CONSOLE, CONSOLE_AND_FILE and FILE. Defaults to 'CONSOLE'. |
|
no |
Maximum duration in milliseconds which is waited if there is a problem
connecting to the ConfigSeeder at startup time. Not setting Defaults to endless retries (not default set). |
|
no |
Duration in milliseconds which is waited between retries if there is a problem connecting to the ConfigSeeder at startup time. Defaults to |
|
no |
Defines if the hostname should be sent to ConfigSeeder as context when looking up configuration values. Defaults to |
The specified keys can be easily translated to environment variable names:
|
4. Modules
4.1. File Module
4.1.1. Introduction
The File Module is responsible for creating different types of configuration files on a Linux or Windows VM.
The following Value Assemblies are used by the file module:
* File: static
: Manage files with static content
* File: template
Manage files which are created with a templating mechanism
4.1.2. Configuration Values within Value Assemblies
Key | Mandatory | Value | ||||
---|---|---|---|---|---|---|
|
(yes) |
Mandatory for Value Assemblies of type Content of the file. If a different file content is required for different environments (or other filter criteria), there could be
|
||||
|
yes |
With this flag, an assembly can be activated or deactivated for different scenarios. To enable an assembly for just environment
To disable an assembly for context
|
||||
|
yes |
Name of the file
If relative or absolut pathes are configured, the correct path separators must be used:
If a file should be provided to a Linux and a Windows VM, there could be the following configuration:
|
||||
|
no |
File mode which should be set:
See Configuration Options for setting a default mode. |
||||
|
no |
Owner of the file, examples:
|
||||
|
no |
Group of the file, examples:
|
||||
|
(yes) |
Mandatory for Value Assemblies of type Content/Template of the file. This content isn’t written directly to a VM, but first sent to the templating service of
ConfigSeeder. Normally, templating is done with Prefix |
||||
|
(yes) |
Mandatory for Value Assemblies of type This value specifies which Configuration Groups are used for the templating. Defaults to the Configuration Group in which the Value Assembly is defined. |
5. Status and Metadata
5.1. Metadata (Stored per File)
5.1.1. Introduction
The OS Connector stores Metadata for every managed file. Depending on the used Operating System, the Metadata is stored in a different way, see later sections.
The following Metadata is required by the OS Connector:
managedByConfigseeder
Boolean flag which defines if a file is managed by ConfigSeeder, meaning if ConfigSeeder OS Connector is allowed to modify or delete the given file. If a ConfigSeeder managed file shouldn’t be modified or deleted by the OS Connector, this flag can be deleted or set to false.
configseederModule
String value which defines which module of the OS Connector is responsible for managing the file.
contentHash Sha256 Hash of the content. Used by the OS Connector to verify if the content of a file has changed since the last run.
This attribute is set when a file is created and updated when the content of a file is updated by OS Connector.
lastUpdatedInMilliseconds
Timestamp which states at which time the file was last updated in ConfigSeeder. This Flag is used to determine if a File is still up to date or if it must be recreated. If the attribute isn’t available, the file will be recreated. This attribute is set or updated to lastUpdatedInMilliseconds of newest ConfigValue at creation and recreation time.
lastUpdatedInMilliseconds is set to the value of the attribute lastUpdateInMilliseconds of the newest ConfigValue which was used to create the file. lastUpdatedInMilliseconds is not set to the time at which the file was modified on the os. |
5.1.2. Linux
Under Linux, the metadata is stored in the extended file attributes, see https://en.wikipedia.org/wiki/Extended_file_attributes#Linux
Medatada | Extended File Attribute |
---|---|
configseeder-Module |
|
contentHash |
|
lastUpdatedInMilliseconds |
|
managedBy-Configseeder |
|
Manually Read or Modify Metadata
Reading Metadata:
$ getfattr -d /tmp/testfile.txt getfattr: Removing leading '/' from absolute path names # file: tmp/testfile.txt user.configseeder-lastUpdatedInMs=0s4YOs3GcBAAA= user.configseeder-managed="true" user.configseeder-module="FILE"
Modifying Metadata:
Do not modify metadata unless you know what you are doing! There should be no reason to modify the metadata
other than parhaps set user.configseeder-managed to false if a file should be removed from ConfigSeeder control
|
$ setfattr /tmp/testfile.txt -n user.configseeder-managed -v "false" $ getfattr -d /tmp/testfile.txt getfattr: Removing leading '/' from absolute path names # file: tmp/testfile.txt user.configseeder-contentHash="8a8f264c0f554ad07c2a6248990ebbaa166b3b00404e8a6dd297ca8715a7859a" user.configseeder-lastUpdatedInMs=0s4YOs3GcBAAA= user.configseeder-managed="false" user.configseeder-module="FILE"
5.1.3. Windows
Under Linux, the metadata is stored as json in an alternate data stream, see https://en.wikipedia.org/wiki/NTFS#Alternate_data_streams_(ADS)
Modifying and saving a file with an editor like Notepad++ or Ultraedit will remove the Alternate Data Stream and the Metadata. |
Manually Read or Modify Metadata
Reading Metadata:
> more < testfile.txt:ConfigseederMetadata {"ManagedByConfigseeder":true,"ConfigSeederModule":"FILE","LastUpdateInMilliseconds":1545587516835,"ContentHash":"8a8f264c0f554ad07c2a6248990ebbaa166b3b00404e8a6dd297ca8715a7859a"}
Writing Metadata:
Do not modify metadata unless you know what you are doing! There should be no reason to modify the metadata
other than perhaps set ManagedByConfigseeder to false if a file should be removed from ConfigSeeder control
|
- Create a File
metadata.txt
with content
{"ManagedByConfigseeder":false,"ConfigSeederModule":"FILE","LastUpdateInMilliseconds":1545587516835,"ContentHash":"8a8f264c0f554ad07c2a6248990ebbaa166b3b00404e8a6dd297ca8715a7859a"}
- Update metadata
> type metadata.txt > testfile.txt:ConfigseederMetadata > more < testfile.txt:ConfigseederMetadata {"ManagedByConfigsee...