Objective
This tutorial walks you through how to install and setup a local Hortonworks Registry to interact with Apache NiFi.
Environment
This tutorial was tested using the following environment and components:
- Mac OS X 10.11.6
- MySQL 5.7.13
- Apache NiFi 1.3.0
- Hortonworks Registry 0.5.4
Note: The record-oriented processors and controller services used in the demo flow of this tutorial were in introduced in NiFi 1.2.0. As such, the tutorial needs to be done running Version 1.2.0 or later. Currently, Hortonworks Registry 0.5.4 is the version compatible with NiFi 1.3.0.
API Guild: http://localhost:9090/swagger
Environment Configuration
Hortonworks Registry Installation
Download the 0.5.4 Registry release:
hortonworks-registry-0.5.4.tar.gz
Extract the tar:
tar xzvf hortonworks-registry-0.5.4.tar.gz
MySQL Database Setup
Login to your MySQL instance and create the schema registry database and necessary users and privileges:
unix> mysql -u root -p unix> Enter password: mysql> create database schema_registry; mysql> CREATE USER 'registry_user'@'localhost' IDENTIFIED BY 'registry_password'; mysql> GRANT ALL PRIVILEGES ON schema_registry.* TO 'registry_user'@'localhost' WITH GRANT OPTION; mysql> commit;
Configure registry.yaml
In the conf directory of the Registry, there is an example MySQL yaml file that we can repurpose:
cd hortonworks-registry-0.5.4 cp conf/registry.yaml.mysql.example conf/registry.yaml
Edit the following section in the yaml file to add appropriate database and user settings:
# MySQL based jdbc provider configuration is: storageProviderConfiguration: providerClass: "com.hortonworks.registries.storage.impl.jdbc.JdbcStorageManager" properties: db.type: "mysql" queryTimeoutInSecs: 30 db.properties: dataSourceClassName: "com.mysql.jdbc.jdbc2.optional.MysqlDataSource" dataSource.url: "jdbc:mysql://localhost/schema_registry" dataSource.user: "registry_user" dataSource.password: "registry_password"
Note: For my environment (with MySQL installed via Homebrew), I did not need to change these default values.
Run Bootstrap Scripts
- ./bootstrap/bootstrap–storage.sh create
Start the Registry Server
- ./bin/registry start
Open Registry UI
Navigate to the registry UI in your browser:
Schema Creation
Select the “+” button to add a schema to the registry:
Configure the schema as follows:
The schema text is:
{ "type": "record", "name": "UserRecord", "fields" : [ {"name": "id", "type": "long"}, {"name": "title", "type": ["null", "string"]}, {"name": "first", "type": ["null", "string"]}, {"name": "last", "type": ["null", "string"]}, {"name": "street", "type": ["null", "string"]}, {"name": "city", "type": ["null", "string"]}, {"name": "state", "type": ["null", "string"]}, {"name": "zip", "type": ["null", "string"]}, {"name": "gender", "type": ["null", "string"]}, {"name": "email", "type": ["null", "string"]}, {"name": "username", "type": ["null", "string"]}, {"name": "password", "type": ["null", "string"]}, {"name": "phone", "type": ["null", "string"]}, {"name": "cell", "type": ["null", "string"]}, {"name": "ssn", "type": ["null", "string"]}, {"name": "date_of_birth", "type": ["null", "string"]}, {"name": "reg_date", "type": ["null", "string"]}, {"name": "large", "type": ["null", "string"]}, {"name": "medium", "type": ["null", "string"]}, {"name": "thumbnail", "type": ["null", "string"]}, {"name": "version", "type": ["null", "string"]}, {"name": "nationality", "type": ["null", "string"]} ] }
Save the schema: