Thursday, September 7, 2017

FIPS-compliant Credential Stores

What is FIPS? FIPS is a U.S. government computer security standard used to approve cryptographic modules [wikipedia].
Was WildFly evaluated for FIPS? No, we are just giving users opportunity to configure and operate Credential Store in compliant way.


To setup Credential Store in FIPS compliant way we need FIPS compliant key store first. This will be used to hold and encryption key and Credential Store implementation will use it to encrypt/decrypt data needed to restore credentials from the storage file.

FIPS 140-2 compliant key store

There are several possibilities to get FIPS compliant key store, this is just on them.
I will use Sun PKCS#11 provider accessing NSS DB. More details lookup here.
Modify $JAVA_HOME/jre/lib/security/ file to have the provider on the first position: /opt/ora/jdk1.8.0_fips/jre/lib/security/nss_pkcs11_fips.cfg
Move all the other providers down one position.

Content of nss_pkcs11_fips.cfg file is following:
name = testPkcs
nssLibraryDirectory = /usr/lib64
nssSecmodDirectory = /home/pskopek/fipsdb
nssDbMode = readWrite
nssModule = fips

Note the nssSecmodDirectory option which points to nssDb directory. You might want to have it elsewhere. Note the name option too.

We need to modify $JAVA_HOME/jre/lib/security/ provider called to use the PKCS#11 key store we are configuring right now. (it is not necessary but might help later when one will try to configure https/ssl).
Resulting row should look like: SunPKCS11-testPkcs

Next step is to initialize NSS DB:
mkdir fipsdb
modutil -force -dbdir fipsdb -create
modutil -force -dbdir fipsdb -fips true
modutil -force -dbdir fipsdb -changepw "NSS FIPS 140-2 Certificate DB"

Just remember to set password "pass123+" as we use it in this example.
Make sure that "fipdb" directory is actually the same like the one referenced by nssSecmodDirectory in nss_pkcs11_fips.cfg configuration file above.

Well that's almost done. We just need to create our secret key to be used later in Credential Store.
We can use Java keytool for the job:
keytool -keystore NONE -storetype PKCS11 -storepass pass123+ -genseckey -alias cskey -keyalg AES -keysize 256
Just be sure you are using the same JRE with file you have already modified.

Credential Store

CLI command to create credential store named "fips".
/subsystem=elytron/credential-store=fips:add(modifiable=true,,, implementation-properties={"keyStoreType"=>"PKCS11","external"=>"true","keyAlias"=>"cskey"},credential-reference={clear-text="pass123+"}, create=true)

We can add some alias to see if it works:
/subsystem=elytron/credential-store=fips:add-alias(alias="myfipsalias", secret-value="supersecret")

Her comes the check:

Thursday, August 17, 2017

Credential Stores in WildFly Managed Domain


Useful reading to start with is WildFly documentation, particularly Domain Setup and Core Management Concepts. To start just download WildFly 11.0.0.Beta1 Application Server Distribution here and you are ready to start.

We will used default configuration as provided in WildFly.
To start server in domain mode run ./bin/ (assuming WildFly home directory is your current one).

Open second terminal window and run ./bin/ --connect to start CLI  for entering commands.

Create Credential Store Resource

In the default WildFly domain configuration we have two servers started "server-one" and "server-two". Both belongs to the "main-server-group" which runs profile "full" (check via CLI:  

We can start then to create credential store named "test" in profile=full.
Here is the command:

Note the /profile=full/ prefix of the resource address. There is no need to define credential store resource at each server. Each server running profile "full" contains our credential store. Therefore it is good idea to locate storage file at server data directory ( as this makes manageable for more servers (and directory is already there).

Runtime operations

As we have successfully defined credential store we can add some aliases through CLI commands. It is similar to standalone server mode, but we have to locate the runtime resource exposed by each individual server.
For our default domain configuration it is following command:
/host=master/server=server-one/subsystem=elytron/credential-store=test:add-alias(alias=db, secret-value="db_secret_password")

This will create alias "db" at server "server-one" at host "master".

/host=master/server=server-two/subsystem=elytron/credential-store=test:add-alias(alias=jms_server, secret-value="jms_secret_password")

This one created alias "jms_server" at "server-two" running at host "master".

Let's check the content of credential stores at each server:

    "outcome" => "success",
    "result" => ["db"]
    "outcome" => "success",
    "result" => ["jms_server"]

Note that content of each credential store is different.
Other operations could be found in this blog post. Just don't forget to locate your server and host to manipulate proper credential store.

Referencing aliases

Each credential store runs at respective server so all credential references used at this sever have to be contained at each credential store. For now we don't support bulk operations across server-groups, so admins must define them for each server individually.
While provisioning another server storage file for credential store can be prepared upfront using WildFly Elytron Tool and distributed to individual server by some secure means. Then it is enough to define credential store and all aliases are available.

Using shared storage file for credential store

One can decide to share one storage file through more servers. There is risk of inconsistency when the credential storage is manipulated from more processes. This is not supported but we cannot really avoid it, so we defined special option in our default credential store implementation called "modifiable" (true/false). Which prevents credential store modifications and therefore user can be sure to control what process can modify the storage.
Note that when implementing your own credential store there is a method which controls this functionality.

Tuesday, July 18, 2017

Custom Test Credential Store

WildFly Elytron Tool - super short introduction

WildFly distribution contains for some time new tool related to Elytron project. Check nightly builds of WildFly 11. Look for bin/ (or other elytron-tool.* scripts for your platform of choice). Detailed blog about features available in Elytron Tool will follow later.
One can start with ./bin/ --help to see what we have there.

Custom Credential Store

Elytron Tool let users to create and use custom implementations of CredentialStoreSpi which can be successfully utilized when one needs to deploy their own credential store which has no support in Elytron.
It simply requires to create implementation class of CredentialStoreSpi and custom JCA provider which will register service for your provider.
I have created simple example project which can be found at GitHub called Custom Test Credential Store. Just build the project with maven (mvn clean install) and you have jar file ready to use.

Elytron Tool example

  1. set your provider jar (and it's dependents) to classpath via
    export ELYTRON_TOOL_ADDONS="/home/pskopek/dev/wildfly-security/custom-test-credential-store/target/custom-test-credential-store-1.0.0-SNAPSHOT.jar"
    In case one needs add more jars use proper file separator for your platform (for Linux ':' or Windows ';')
  2. use script to invoke Elytron Tool as invoking through "java -jar ..." cannot add anything to classpath. There are scripts for different platforms.
  3. Try following command to create your custom credential store and add alias "s2" for secret "DDD"
    ./bin/ credential-store --create --location --password "secret2" --summary --add s2 --secret DDD --type CustomTestCredentialStore --salt AQWSDERF --iteration 23 
  4. Note "--type CustomTestCredentialStore" option to actually use custom implementation instead of the default one. This option must be present anytime when one which to manipulate custom credential store.

Tuesday, June 6, 2017

Update of Credential Store alias operations

Recent updates of Elytron subsystem in WildFly core contains reworked operations on aliases stored inside Credential Store. Older approach with having aliases as resources was not ideal, so from now on we use operations on credential-store resource.
Let me give you some examples:

Create credential store named "test":
 /subsystem=elytron/credential-store=test:add(,, credential-reference={clear-text="secret2"}, create=true)  

Add alias "my_db_password" with secret value "supersecret":
/subsystem=elytron/credential-store=test:add-alias(alias="my_db_password", secret-value="supersecret")

Check presence of aliases:

Change secret value attached to an alias:
/subsystem=elytron/credential-store=test:set-secret(alias="my_db_password", secret-value="notsosecret")

There is a reload operation which can be used in case the storage file was modified by outside process. Credential store "test" has to be reloaded the change can take the effect:

Finally we can demonstrate removal of the alias from credential store "test":

Tuesday, May 16, 2017

Credential References

The last post showed how to create credential store and manage aliases of credentials there. We will continue down the road to show how to use them.

What is Credential Reference?

Credential reference is new type of attribute one can find in management model of WildFly and API to work with it. Credential reference is object type attribute consisting of several simple attributes:
  • credential store - name of credential store)
  • credential alias - name of the alias
  • credential type - type of credential the reference denotes
  • clear text - simple clear text attribute to be used as password credential or interpreted as command from underlying OS

How Credential Reference Works?

For developers: look at utility class it contains definitions of attribute and builders ready to use. The class is located at wildfly-core module controller.
The key part is method getCredentialSourceSupplier which returns in fact returns CredentialSource supplier with Exception (ExceptionSupplier<CredentialSource, Exception>). CredentialSource is used later to resolve credential using one of its getCredential methods.

For administrators: use credential-reference attribute on resources where you want or need to hide clear text passwords from configuration (you should do it in production systems anyway ;-) ).

Through Credential Store

Define your credential store (suppose we have the definition from older blog post).
First thing first, create your alias

Let's add new data source using this command
data-source add --name=testDS --jndi-name=java:/testDS --driver-name=h2 --connection-url=jdbc:h2:mem:test;DB_CLOSE_DELAY=-1;DB_CLOSE_ON_EXIT=FALSE --user-name=sa --credential-reference={store=test, alias=test_db_password}

We can check the result now with command /subsystem=datasources/data-source=testDS:read-resource(recursive=true)
    "outcome" => "success",
    "result" => {
        "allocation-retry" => undefined,
        "allocation-retry-wait-millis" => undefined,
        "allow-multiple-users" => false,
        "authentication-context" => undefined,
        "background-validation" => undefined,
        "background-validation-millis" => undefined,
        "blocking-timeout-wait-millis" => undefined,
        "capacity-decrementer-class" => undefined,
        "capacity-decrementer-properties" => undefined,
        "capacity-incrementer-class" => undefined,
        "capacity-incrementer-properties" => undefined,
        "check-valid-connection-sql" => undefined,
        "connectable" => false,
        "connection-listener-class" => undefined,
        "connection-listener-property" => undefined,
        "connection-url" => "jdbc:h2:mem:test;DB_CLOSE_DELAY=-1;DB_CLOSE_ON_EXIT=FALSE",
        "credential-reference" => {
            "store" => "test",
            "alias" => "test_db_password"
        "datasource-class" => undefined,
        "driver-class" => undefined,
        "driver-name" => "h2",
        "elytron-enabled" => false,
        "enabled" => true,
        "enlistment-trace" => false,
        "exception-sorter-class-name" => undefined,
        "exception-sorter-properties" => undefined,
        "flush-strategy" => undefined,
        "idle-timeout-minutes" => undefined,
        "initial-pool-size" => undefined,
        "jndi-name" => "java:/testDS",
        "jta" => true,
        "max-pool-size" => undefined,
        "mcp" => "org.jboss.jca.core.connectionmanager.pool.mcp.SemaphoreConcurrentLinkedDequeManagedConnectionPool",
        "min-pool-size" => undefined,
        "new-connection-sql" => undefined,
        "password" => undefined,
        "pool-fair" => undefined,
        "pool-prefill" => undefined,
        "pool-use-strict-min" => undefined,
        "prepared-statements-cache-size" => undefined,
        "query-timeout" => undefined,
        "reauth-plugin-class-name" => undefined,
        "reauth-plugin-properties" => undefined,
        "security-domain" => undefined,
        "set-tx-query-timeout" => false,
        "share-prepared-statements" => false,
        "spy" => false,
        "stale-connection-checker-class-name" => undefined,
        "stale-connection-checker-properties" => undefined,
        "statistics-enabled" => false,
        "track-statements" => "NOWARN",
        "tracking" => false,
        "transaction-isolation" => undefined,
        "url-delimiter" => undefined,
        "url-selector-strategy-class-name" => undefined,
        "use-ccm" => true,
        "use-fast-fail" => false,
        "use-java-context" => true,
        "use-try-lock" => undefined,
        "user-name" => "sa",
        "valid-connection-checker-class-name" => undefined,
        "valid-connection-checker-properties" => undefined,
        "validate-on-match" => undefined,
        "connection-properties" => undefined,
        "statistics" => undefined
Note the credential-reference attribute and password is undefined.

Simple masked password credential

It is credential reference compatible with older PicketBox Vault masked password hash which can be generated using WildFly Elytron Tool (peek to $JBOSS_HOME/bin directory or wait for next blog post).
Create mask string:
./bin/ mask -i 351 -s A1B2C3D4 -x sa

Credential reference attribute syntax:

OS command supplied password credential

There are use cases when it is reasonable to get password credential from underlying OS command (read it from standard output). For this we have another for of credential-reference. We are supporting two types of syntax. Space delimited and comma delimited command line arguments.
Space delimited:
credential-reference={clear-text="{EXT}/usr/bin/ arg1 arg2 arg3", type="COMMAND"}

Comma delimited:
credential-reference={clear-text="{CMD}/usr/bin/,arg1,arg2,arg3", type="COMMAND"}

Directly specified clear text password credential

There also the simplest way to specify password credential without anyhow obscuring the value. Use with caution.

Note on security

All mentioned obscuring cases are not bullet proof and there should be further means to make it harder to reveal the credential from configuration.

Tuesday, April 25, 2017

Basics of Credential Store in WildFly (11.0.0.Beta1 Nightly)


Credential store came to life as follower of PicketBox Security Vault with some extended capabilities and better integration to the new WildFly security framework (WildFly Elytron).
More about Elytron could be found here.
The best way to test credential store functionality is to grab latest nightly build of WildFly 11 and take for a ride.

How to create credential store

One can manipulate credential store using CLI provided by WildFly or use Elytron Tool (part of WildFly) to manipulate credential store "offline" and then use it in your WildFly instance.

Create credential store in CLI

Start WildFly standalone server using ./bin/ Open command line interface using
./bin/ --connect
There should already be installed Elytron Subsystem which can be easily check using CLI command
ls /subsystem=elytron
Creating credential store is as easy as running this CLI command:
 /subsystem=elytron/credential-store=test:add(,, credential-reference={clear-text="secret2"}, create=true)  
This command adds credential store named "test" with storage file named in data directory of your WildFly instance (ls ./standalone/data/). Default implementation of credential store will not create storage file until you add the first credential to the store.
Credentials are stored under alias which they can be later referred to in your configuration.
This command adds alias "my_db_password" with value "supersecret" as the password itself.

The storage file "" is created right now.

Credential store manipulation

Note: manipulation style has changed, see more here.

There are several manipulation commands in CLI one can use to create credential store content.
Add new alias:
Remove from credential store alias:
Show all aliases:
ls /subsystem=elytron/credential-store=test/alias=

Reference your credential alias

To use your credential alias from credential store we introduced credential-reference attribute. One can find credential references all over the model. We can demonstrate its usage at credential store itself. Let's create new credential store with guarded with password "supersecret" as created above.
/subsystem=elytron/credential-store=another-test:add(, credential-reference={store=test, alias=my_db_password}, create=true)
Notice credential-reference attribute using store and alias to point to different credential store (test) to fetch real password value.
Let's test if it functions properly and store some alias in the new store.
We have two credential stores at the moment (test and another-test):
ls /subsystem=elytron/credential-store=
Credential reference supports also other types of references which I will cover in one of next posts.