Uploaded image for project: 'Hadoop Common'
  1. Hadoop Common
  2. HADOOP-18709

Add curator based ZooKeeper communication support over SSL/TLS into the common library

VotersWatch issueWatchersCreate sub-taskLinkCloneUpdate Comment AuthorReplace String in CommentUpdate Comment VisibilityDelete Comments
    XMLWordPrintableJSON

Details

    • Improvement
    • Status: Resolved
    • Major
    • Resolution: Fixed
    • 3.4.0
    • 3.4.0
    • common
    • Reviewed

    Description

      With HADOOP-16579 the ZooKeeper client is capable of securing communication with SSL. 

      To follow the convention introduced in HADOOP-14741, proposing to add to the core-default.xml the following configurations, as the groundwork for the components to enable encrypted communication between the individual components and ZooKeeper:

      • hadoop.zk.ssl.keystore.location
      • hadoop.zk.ssl.keystore.password
      • hadoop.zk.ssl.truststore.location
      • hadoop.zk.ssl.truststore.password

      These parameters along with the component-specific ssl.client.enable option (e.g. yarn.zookeeper.ssl.client.enable) should be passed to the ZKCuratorManager to build the CuratorFramework. The ZKCuratorManager needs a new overloaded start() method to build the encrypted communication.

      • The secured ZK Client uses Netty, hence the dependency is included in the pom.xml. Added netty-handler and netty-transport-native-epoll dependency to the pom.xml based on ZOOKEEPER-3494 - "No need to depend on netty-all (SSL)".
      • The change was exclusively tested with the unit test, which is a kind of integration test, as a ZK Server was brought up and the communication tested between the client and the server.
      • This code change is in the common code base and there is no component calling it yet. Once YARN-11468 - "Zookeeper SSL/TLS support" is implemented, we can test it in a real cluster environment.

      Attachments

        Issue Links

        Activity

          This comment will be Viewable by All Users Viewable by All Users
          Cancel

          People

            bender Ferenc Erdelyi
            bender Ferenc Erdelyi
            Votes:
            0 Vote for this issue
            Watchers:
            3 Start watching this issue

            Dates

              Created:
              Updated:
              Resolved:

              Slack

                Issue deployment