BatchIQ instances now include NiFi-Init, a boot-time configuration tool for easily creating secure and repeatable Apache NiFi installations from YAML templates. NiFi-Init was inspired by the popular Cloud-Init tool for general-purpose instance configuration. Obviously, NiFi-Init is specific to Apache NiFi, with a template grammar customized to NiFi configuration options and requirements. Templates may be provided through instance User Data or URLs to HTTP, HTTPS, and S3 resources.
The initial feature set targets security configuration, making it easy to deploy a secured NiFi installation using SSL/X.509 certificates, or just plain username/password authentication. Username/password authentication might be as simple as this:
#nifi-init name: NiFi with Simple Username/Password Security security: scheme: simple-username-password users: james: roles: ROLE_DFM, ROLE_PROVENANCE password: $2a$10$m/BtaawA/1yI2zacRZtQ2OUdC/ANnb08lYMyU1dKRMxfDIjGNplUp
Where NiFi is provisioned to let me log in with a password, specified in Bcrypt hash format in the template. For SSL/X.509 security, we might want to start up NiFi with our company's Certificate Authority configured in the Truststore:
#nifi-init name: NiFi with SSL Security security: scheme: ssl keystore: auto truststore: certificates: my_company_ca: s3://mybucket/certs/my_company_ca.pem authorization: 'CN=james, OU=test, O=TestCo, L=Seattle, ST=WA, C=US': ROLE_ADMIN, ROLE_DFM, ROLE_PROVENANCE aws: cloudwatch-logs: true notify: sns: topic-arn: arn:aws:sns:us-east-1:123456789012:NiFi_Operators
The template above configures NiFi security with a self-signed Keystore, and build a Truststore using our company CA certificate. It sets up authorization for a user to log in and start using NiFi. It turns on the CloudWatch Logs agent to capture NiFi logs. And it notifies us when NiFi is ready.
That's pretty cool, and even cooler that we can check these templates into source control and recreate a predicatable NiFi configuration. Or, if you store templates on S3, you can reference them directly:
Ready to Try It?
You can get started with NiFi-Init by launching a BatchIQ instance from the AWS Marketplace. NiFi-Init is supported in the most recent version, 0.6.1-1. Templates must be included in the instance User Data at launch.
In the AWS Marketplace, select the "Manual Launch" option to use the familiar EC2 Launch Instance flow.
While configuring the launch settings, you will find User Data under Advanced Details on the Configure Instance page.
Please see the NiFi-Init User Guide and Reference for more on specific features. I am looking forward to hearing how NiFi-Init works for you, and what features can be added to make it more useful.