Installing a local Hortonworks Registry to use

https://community.hortonworks.com/articles/119766/installing-a-local-hortonworks-registry-to-use-wit.html

Objective

This tutorial walks you through how to install and setup a local Hortonworks Registry to interact with Apache NiFi.

Environment

This tutorial was tested using the following environment and components:

  • Mac OS X 10.11.6
  • MySQL 5.7.13
  • Apache NiFi 1.3.0
  • Hortonworks Registry 0.5.4

Note: The record-oriented processors and controller services used in the demo flow of this tutorial were in introduced in NiFi 1.2.0. As such, the tutorial needs to be done running Version 1.2.0 or later. Currently, Hortonworks Registry 0.5.4 is the version compatible with NiFi 1.3.0.

API Guild: http://localhost:9090/swagger

Environment Configuration

Hortonworks Registry Installation

Download the 0.5.4 Registry release:

hortonworks-registry-0.5.4.tar.gz

Extract the tar:

tar xzvf hortonworks-registry-0.5.4.tar.gz

MySQL Database Setup

Login to your MySQL instance and create the schema registry database and necessary users and privileges:

unix> mysql -u root -p
unix> Enter password:
mysql> create database schema_registry;
mysql> CREATE USER 'registry_user'@'localhost' IDENTIFIED BY 'registry_password';
mysql> GRANT ALL PRIVILEGES ON schema_registry.* TO 'registry_user'@'localhost' WITH GRANT OPTION;
mysql> commit;

Configure registry.yaml

In the conf directory of the Registry, there is an example MySQL yaml file that we can repurpose:

cd hortonworks-registry-0.5.4
cp conf/registry.yaml.mysql.example conf/registry.yaml

Edit the following section in the yaml file to add appropriate database and user settings:

# MySQL based jdbc provider configuration is:
storageProviderConfiguration:
 providerClass: "com.hortonworks.registries.storage.impl.jdbc.JdbcStorageManager"
 properties:
   db.type: "mysql"
   queryTimeoutInSecs: 30
   db.properties:
     dataSourceClassName: "com.mysql.jdbc.jdbc2.optional.MysqlDataSource"
     dataSource.url: "jdbc:mysql://localhost/schema_registry"
     dataSource.user: "registry_user"
     dataSource.password: "registry_password"

Note: For my environment (with MySQL installed via Homebrew), I did not need to change these default values.

Run Bootstrap Scripts

  1. ./bootstrap/bootstrapstorage.sh create

Start the Registry Server

  1. ./bin/registry start

Open Registry UI

Navigate to the registry UI in your browser:

http://localhost:9090

Schema Creation

Select the “+” button to add a schema to the registry:

Configure the schema as follows:

The schema text is:

{
"type": "record",
"name": "UserRecord",
"fields" : [
	{"name": "id", "type": "long"},
	{"name": "title", "type": ["null", "string"]},
	{"name": "first", "type": ["null", "string"]},
	{"name": "last", "type": ["null", "string"]},
	{"name": "street", "type": ["null", "string"]},
	{"name": "city", "type": ["null", "string"]},
	{"name": "state", "type": ["null", "string"]},
	{"name": "zip", "type": ["null", "string"]},
	{"name": "gender", "type": ["null", "string"]},
	{"name": "email", "type": ["null", "string"]},
	{"name": "username", "type": ["null", "string"]},
	{"name": "password", "type": ["null", "string"]},
	{"name": "phone", "type": ["null", "string"]},
	{"name": "cell", "type": ["null", "string"]},
	{"name": "ssn", "type": ["null", "string"]},
	{"name": "date_of_birth", "type": ["null", "string"]},
	{"name": "reg_date", "type": ["null", "string"]},
	{"name": "large", "type": ["null", "string"]},
	{"name": "medium", "type": ["null", "string"]},
	{"name": "thumbnail", "type": ["null", "string"]},
	{"name": "version", "type": ["null", "string"]},
	{"name": "nationality", "type": ["null", "string"]}
  ]
}

Save the schema:

 

Leave a Reply

Your email address will not be published. Required fields are marked *