Hive Accessing BigData

hive hadoop
8 Sep 2020

Hive Access BigData

//
Comments0

Hive is a data warehouse infrastructure tool to process structured data in Hadoop. It resides on top of Hadoop to summarize Big Data, and makes querying and analyzing easy.This article provides a brief on how to Hive Access BigData environment with ambari configuration

Apache Knox:

Apache Knox is the open source project under Apache Software Foundation similar to most other Hadoop Ecosystem projects. It provides a REST API Gateway for interacting with Apache Hadoop clusters. When using Knox Gateway with the Hadoop cluster it will be the single access point for all REST interactions with the clusters.

Following are the basic functionality provided by Knox:

  • Authentication (LDAP and Active Directory Authentication Provider)
  • Federation/SSO (HTTP Header Based Identity Federation)
  • Authorization (Service Level Authorization)
  • Auditing

Apache Knox is already compatible with a lot of Apache Ecosystem projects and provides integration with the following services:

  • Ambari
  • Hadoop Distributed File System
  • Templeton (HCatalog)
  • Stargate (HBase)
  • Oozie
  • Hive/JDBC
  • Yarn RM
  • Storm

Besides this Apache Knox also supports following Apache Hadoop UIs:

  • Name Node UI
  • Job History UI
  • Oozie UI
  • HBase UI
  • Yarn UI
  • Spark UI
  • Ambari UI
  • Ranger Admin Console

Setup Knox:

Access the Hive:

  • Change in Ambari the Hive config hive.server2.transport.mode to HTTP from binary
  • Check-in Linux
  • Use Commands
  • beeline
!connect jdbc:hive2://hadoopmaster.com:8443/;
ssl=true;sslTrustStore=/var/lib/knox/data-2.6.1.0–129/security/keystores/gateway.jks;
[email protected]?hive.server2.transport.mode=http;
hive.server2.thrift.http.path=gateway/default/hive

  • hadoopmaster.com is your system address
  • 8443 is your port to Knox
  • sslTrustStore change according to the Knox version
  • trustStorePassword is the password used during setup
  • Enter username guest
  • Enter password guest-password
  • (The above username and password is from the Advance Topology file in config)
  • Access the HDFS:
  • In Linux
  • curl -i -k -u guest:guest-password -X GET \ ‘https://<KnoxHostAddress>:8443/gateway/<ClusterName>/webhdfs/v1/?op=LISTSTATUS’
  • Browser
  • https://hadoopmaster.gmail.com:8443/gateway/default/webhdfs/v1/?op=GETFILESTATUS
  • Provide same credential guest and guest-password when prompt

In this quick tutorial, we started with the introduction of Apache Knox and what are the components of the Hadoop Ecosystem which it covers and support. We went through the installation process of the Apache Knox in the subsequent sections of the tutorial.

Leave a Reply

/home/b0eee662nkt0/public_html/wp-content/themes/clevercourse/single.php