645三.创建文件夹,手动放入solr,hbase(不然maven下载时间较长)3.10.8 6522.9.2
/opt/lagou/servers/apache-atlas-sources-1.2.0/distro [root@Linux122 distro]# mkdir solr [root@Linux122 distro]# mkdir hbase guchenfeideMacBook-Pro:Downloads guchenfei$ scp hbase-1.1.2.tar.gz root@Linux122:/opt/lagou/servers/apache-atlas-sources-1.2.0/distro/hbase guchenfeideMacBook-Pro:Downloads guchenfei$ scp solr-5.5.1.tgz root@Linux122:/opt/lagou/servers/apache-atlas-sources-1.2.0/distro/solr四.利用maven编译
atlas路径下 export MAVEN_OPTS="-Xms2g -Xmx2g" mvn clean -DskipTests package -Pdist,embeded-hbase-solr
编译过程中遇到以下错误
报错1
[ERROR] Failed to execute goal on project atlas-graphdb-janus: Could not resolve dependencies for project org.apache.atlas:atlas-graphdb-janus:jar:1.2.0: Failure to find com.sleepycat:je:jar:7.4.5 in http://maven.aliyun.com/nexus/content/groups/public/ was cached in the local repository, resolution will not be reattempted until the update interval of nexus has elapsed or updates are forced -> [Help 1] [ERROR] [ERROR] To see the full stack trace of the errors, re-run Maven with the -e switch. [ERROR] Re-run Maven using the -X switch to enable full debug logging. [ERROR] [ERROR] For more information about the errors and possible solutions, please read the following articles: [ERROR] [Help 1] http://cwiki.apache.org/confluence/display/MAVEN/DependencyResolutionException [ERROR] [ERROR] After correcting the problems, you can resume the build with the command [ERROR] mvn -rf :atlas-graphdb-janus
在atlas文件下修改配置
pom.xml
oracleReleases Oracle Released Java Packages http://download.oracle.com/maven default
setting.xml
oracleReleases oracleReleases Oracle Released Java Packages http://download.oracle.com/maven
报错2
[WARNING] The requested profile "embeded-hbase-solr" could not be activated because it does not exist. [ERROR] Failed to execute goal on project hive-bridge-shim: Could not resolve dependencies for project org.apache.atlas:hive-bridge-shim:jar:1.2.0: Could not find artifact org.pentaho:pentaho-aggdesigner-algorithm:jar:5.1.5-jhyde in nexus (http://maven.aliyun.com/nexus/content/groups/public/) -> [Help 1] [ERROR] [ERROR] To see the full stack trace of the errors, re-run Maven with the -e switch. [ERROR] Re-run Maven using the -X switch to enable full debug logging. [ERROR] [ERROR] For more information about the errors and possible solutions, please read the following articles: [ERROR] [Help 1] http://cwiki.apache.org/confluence/display/MAVEN/DependencyResolutionException [ERROR] [ERROR] After correcting the problems, you can resume the build with the command [ERROR] mvn -rf :hive-bridge-shim
原因：引用jar包找不到
解决方法：从如下网址下载jar，上传到maven本地库路径
路径:org/pentaho/pentaho-aggdesigner-algorithm/5.1.5-jhyde
https://public.nexus.pentaho.org/content/groups/omni/org/pentaho/pentaho-aggdesigner-algorithm/5.1.5-jhyde/
注意下载正确的jar包
报错3
[WARNING] The requested profile "embeded-hbase-solr" could not be activated because it does not exist. [ERROR] Failed to execute goal on project sqoop-bridge-shim: Could not resolve dependencies for project org.apache.atlas:sqoop-bridge-shim:jar:1.2.0: Could not find artifact org.apache.sqoop:sqoop:jar:1.4.6.2.3.99.0-195 in nexus (http://maven.aliyun.com/nexus/content/groups/public/) -> [Help 1] [ERROR] [ERROR] To see the full stack trace of the errors, re-run Maven with the -e switch. [ERROR] Re-run Maven using the -X switch to enable full debug logging. [ERROR] [ERROR] For more information about the errors and possible solutions, please read the following articles: [ERROR] [Help 1] http://cwiki.apache.org/confluence/display/MAVEN/DependencyResolutionException [ERROR] [ERROR] After correcting the problems, you can resume the build with the command [ERROR] mvn -rf :sqoop-bridge-shim
同样下载jar包放到以下路径
guchenfeideMacBook-Pro:Downloads guchenfei$ scp sqoop-1.4.6.2.3.99.0-195.jar root@Linux122:/root/.m2/repository/org/apache/sqoop/sqoop/1.4.6.2.3.99.0-195
编译成功
[INFO] Reactor Summary: [INFO] [INFO] Apache Atlas Server Build Tools 1.0 ................ SUCCESS [ 6.071 s] [INFO] apache-atlas 1.2.0 ................................. SUCCESS [ 23.063 s] [INFO] Apache Atlas Test Utility Tools 1.2.0 .............. SUCCESS [ 18.065 s] [INFO] Apache Atlas Integration 1.2.0 ..................... SUCCESS [ 13.976 s] [INFO] Apache Atlas Common 1.2.0 .......................... SUCCESS [ 4.861 s] [INFO] Apache Atlas Client 1.2.0 .......................... SUCCESS [ 0.704 s] [INFO] atlas-client-common 1.2.0 .......................... SUCCESS [ 2.351 s] [INFO] atlas-client-v1 1.2.0 .............................. SUCCESS [ 3.047 s] [INFO] Apache Atlas Server API 1.2.0 ...................... SUCCESS [ 3.021 s] [INFO] Apache Atlas Notification 1.2.0 .................... SUCCESS [ 8.389 s] [INFO] atlas-client-v2 1.2.0 .............................. SUCCESS [ 2.216 s] [INFO] Apache Atlas Graph Database Projects 1.2.0 ......... SUCCESS [ 0.511 s] [INFO] Apache Atlas Graph Database API 1.2.0 .............. SUCCESS [ 2.738 s] [INFO] Graph Database Common Code 1.2.0 ................... SUCCESS [ 2.501 s] [INFO] Apache Atlas JanusGraph DB Impl 1.2.0 .............. SUCCESS [ 17.061 s] [INFO] Apache Atlas Graph Database Implementation Dependencies 1.2.0 SUCCESS [ 3.629 s] [INFO] Shaded version of Apache hbase client 1.2.0 ........ SUCCESS [ 13.936 s] [INFO] Shaded version of Apache hbase server 1.2.0 ........ SUCCESS [ 36.452 s] [INFO] Apache Atlas Authorization 1.2.0 ................... SUCCESS [ 3.678 s] [INFO] Apache Atlas Repository 1.2.0 ...................... SUCCESS [ 23.099 s] [INFO] Apache Atlas UI 1.2.0 .............................. SUCCESS [04:08 min] [INFO] Apache Atlas Web Application 1.2.0 ................. SUCCESS [06:59 min] [INFO] Apache Atlas documentation 1.2.0 ................... SUCCESS [ 33.017 s] [INFO] Apache Atlas FileSystem Model 1.2.0 ................ SUCCESS [ 18.675 s] [INFO] Apache Atlas Plugin Classloader 1.2.0 .............. SUCCESS [ 2.905 s] [INFO] Apache Atlas Hive Bridge Shim 1.2.0 ................ SUCCESS [ 13.538 s] [INFO] Apache Atlas Hive Bridge 1.2.0 ..................... SUCCESS [ 43.045 s] [INFO] Apache Atlas Falcon Bridge Shim 1.2.0 .............. SUCCESS [ 9.073 s] [INFO] Apache Atlas Falcon Bridge 1.2.0 ................... SUCCESS [ 14.093 s] [INFO] Apache Atlas Sqoop Bridge Shim 1.2.0 ............... SUCCESS [ 11.648 s] [INFO] Apache Atlas Sqoop Bridge 1.2.0 .................... SUCCESS [ 16.375 s] [INFO] Apache Atlas Storm Bridge Shim 1.2.0 ............... SUCCESS [ 18.567 s] [INFO] Apache Atlas Storm Bridge 1.2.0 .................... SUCCESS [ 26.652 s] [INFO] Apache Atlas Hbase Bridge Shim 1.2.0 ............... SUCCESS [ 9.138 s] [INFO] Apache Atlas Hbase Bridge 1.2.0 .................... SUCCESS [ 33.532 s] [INFO] Apache Atlas Kafka Bridge 1.2.0 .................... SUCCESS [ 12.331 s] [INFO] Apache Atlas Distribution 1.2.0 .................... SUCCESS [01:53 min] [INFO] ------------------------------------------------------------------------ [INFO] BUILD SUCCESS [INFO] ------------------------------------------------------------------------ [INFO] Total time: 20:40 min [INFO] Finished at: 2022-01-30T19:43:07+08:00 [INFO] ------------------------------------------------------------------------ [WARNING] The requested profile "embeded-hbase-solr" could not be activated because it does not exist. [root@Linux122 apache-atlas-sources-1.2.0]# ls五.安装
进入编译产物目录下
/opt/lagou/servers/apache-atlas-sources-1.2.0/distro/target
将apache-atlas-1.2.0-bin.tar.gz copy到安装目录
[root@Linux122 target]# cp apache-atlas-1.2.0-bin.tar.gz /opt/lagou/servers/
解压
[root@Linux122 servers]# tar -zxvf apache-atlas-1.2.0-bin.tar.gz [root@Linux122 servers]# mv apache-atlas-1.2.0 atlas-1.2.0六.配置环境变量
export ATLAS_HOME=/opt/lagou/servers/atlas-1.2.0 export PATH=$PATH:$ATLAS_HOME/bin /opt/lagou/servers/atlas-1.2.0/conf atlas-env.sh 添加hbase的配置 #Hbase配置文件路径 export Hbase_CONF_DIR=/opt/lagou/servers/atlas-1.2.0/conf/hbase/hbase-site.xml 记得在对应路径下将模版文件改成配置文件七.启动服务
执行bin下脚本 [root@Linux122 bin]# ./atlas_start.py starting atlas on host localhost starting atlas on port 21000 ..................................................................................................... Apache Atlas Server started!!!
查看后台进程
[root@Linux122 bin]# ps -ef | grep atlas root 7576 1 2 21:58 pts/0 00:00:13 /opt/lagou/servers/jdk1.8.0_301/bin/java -Datlas.log.dir=/opt/lagou/servers/atlas-1.2.0/logs -Datlas.log.file=application.log -Datlas.home=/opt/lagou/servers/atlas-1.2.0 -Datlas.conf=/opt/lagou/servers/atlas-1.2.0/conf -Xmx1024m -Dlog4j.configuration=atlas-log4j.xml -Djava.net.preferIPv4Stack=true -server -classpath /opt/lagou/servers/atlas-1.2.0/conf:/opt/lagou/servers/atlas-1.2.0/server/webapp/atlas/WEB-INF/classes:/opt/lagou/servers/atlas-1.2.0/server/webapp/atlas/WEB-INF/lib/*:/opt/lagou/servers/atlas-1.2.0/libext/*:/opt/lagou/servers/atlas-1.2.0/conf/hbase/hbase-site.xml org.apache.atlas.Atlas -app /opt/lagou/servers/atlas-1.2.0/server/webapp/atlas root 7954 3409 1 22:06 pts/0 00:00:00 grep --color=auto atlas
通过查看启动的进程发现缺少其他服务
于是重新进行了编译
mvn clean -DskipTests install mvn clean -DskipTests package -Pdist,embedded-hbase-solr
剩余工作一样
重新启动服务
[root@Linux122 bin]# ./atlas_start.py configured for local hbase. hbase started. configured for local solr. solr started. setting up solr collections... starting atlas on host localhost starting atlas on port 21000 .............................................................................................................................................................................................................................................................................................................. Apache Atlas Server started!!!
查看进程
[root@Linux122 bin]# ps -ef | grep atlas root 7013 1 0 16:59 pts/0 00:00:00 bash /opt/lagou/servers/atlas-1.2.0/hbase/bin/hbase-daemon.sh --config /opt/lagou/servers/atlas-1.2.0/hbase/conf foreground_start master root 7107 1 5 16:59 pts/0 00:00:23 /opt/lagou/servers/jdk1.8.0_301/bin/java -server -Xms512m -Xmx512m -XX:NewRatio=3 -XX:SurvivorRatio=4 -XX:TargetSurvivorRatio=90 -XX:MaxTenuringThreshold=8 -XX:+UseConcMarkSweepGC -XX:+UseParNewGC -XX:ConcGCThreads=4 -XX:ParallelGCThreads=4 -XX:+CMSScavengeBeforeRemark -XX:PretenureSizeThreshold=64m -XX:+UseCMSInitiatingOccupancyOnly -XX:CMSInitiatingOccupancyFraction=50 -XX:CMSMaxAbortablePrecleanTime=6000 -XX:+CMSParallelRemarkEnabled -XX:+ParallelRefProcEnabled -verbose:gc -XX:+PrintHeapAtGC -XX:+PrintGCDetails -XX:+PrintGCDateStamps -XX:+PrintGCTimeStamps -XX:+PrintTenuringDistribution -XX:+PrintGCApplicationStoppedTime -Xloggc:/opt/lagou/servers/atlas-1.2.0/solr/server/logs/solr_gc.log -DzkClientTimeout=15000 -DzkHost=localhost:2181 -Djetty.port=9838 -DSTOP.PORT=8838 -DSTOP.KEY=solrrocks -Duser.timezone=UTC -Djetty.home=/opt/lagou/servers/atlas-1.2.0/solr/server -Dsolr.solr.home=/opt/lagou/servers/atlas-1.2.0/solr/server/solr -Dsolr.install.dir=/opt/lagou/servers/atlas-1.2.0/solr -Xss256k -XX:OnOutOfMemoryError=/opt/lagou/servers/atlas-1.2.0/solr/bin/oom_solr.sh 9838 /opt/lagou/servers/atlas-1.2.0/solr/server/logs -jar start.jar --module=http root 7767 1 12 17:00 pts/0 00:00:47 /opt/lagou/servers/jdk1.8.0_301/bin/java -Datlas.log.dir=/opt/lagou/servers/atlas-1.2.0/logs -Datlas.log.file=application.log -Datlas.home=/opt/lagou/servers/atlas-1.2.0 -Datlas.conf=/opt/lagou/servers/atlas-1.2.0/conf -Xmx1024m -Dlog4j.configuration=atlas-log4j.xml -Djava.net.preferIPv4Stack=true -server -classpath /opt/lagou/servers/atlas-1.2.0/conf:/opt/lagou/servers/atlas-1.2.0/server/webapp/atlas/WEB-INF/classes:/opt/lagou/servers/atlas-1.2.0/server/webapp/atlas/WEB-INF/lib/*:/opt/lagou/servers/atlas-1.2.0/libext/*:/opt/lagou/servers/atlas-1.2.0/hbase/conf org.apache.atlas.Atlas -app /opt/lagou/servers/atlas-1.2.0/server/webapp/atlas root 8796 3249 0 17:06 pts/0 00:00:00 grep --color=auto atlas
启动成功
访问网址
http://Linux122:21000/login.jsp
用缺省的账户密码:admin,admin登录
自定义用户名和密码
/opt/lagou/servers/atlas-1.2.0/conf [root@Linux122 conf]# more users-credentials.properties #username=group::sha256-password admin=ADMIN::8c6976e5b5410415bde908bd4dee15dfb167a9c873fc4bb8a81f6f2ab448a918 rangertagsync=RANGER_TAG_SYNC::e3f67240f5117d1753c940dae9eea772d36ed5fe9bd9c94a300e4041 3f1afb9d
其中
ADMIN::8c6976e5b5410415bde908bd4dee15dfb167a9c873fc4bb8a81f6f2ab448a918这个是账号和密码 [root@Linux122 conf]# echo -n "admin" | sha256sum 8c6976e5b5410415bde908bd4dee15dfb167a9c873fc4bb8a81f6f2ab448a918 -
修改密码可以将自己的自定义密码改成计算后的sha256sum值
八.Hive血缘关系导入 1.配置hive环境变量,将/opt/lagou/servers/atlas-1.2.0/conf/atlas-application.properties拷贝到/opt/lagou/servers/hive-2.3.7/conf让hive元数据变更时候告诉atlas软连接和硬拷贝都可以
ln -s /opt/lagou/servers/atlas-1.2.0/conf/atlas-application.properties /opt/lagou/servers/hive-2.3.7/conf2.将json解析相关的jar包拷贝
/opt/lagou/servers/atlas-1.2.0/server/webapp/atlas/WEB-INF/lib [root@Linux122 lib]# ls jackson* jackson-annotations-2.9.9.jar jackson-jaxrs-base-2.9.9.jar jackson-core-2.9.9.jar jackson-jaxrs-json-provider-2.9.9.jar jackson-core-asl-1.9.2.jar jackson-mapper-asl-1.9.2.jar jackson-databind-2.7.8.jar jackson-module-jaxb-annotations-2.9.9.jar jackson-dataformat-smile-2.5.4.jar
将以下jar包软连接到/opt/lagou/servers/atlas-1.2.0/hook/hive/atlas-hive-plugin-impl
[root@Linux122 lib]# ln -s /opt/lagou/servers/atlas-1.2.0/server/webapp/atlas/WEB-INF/lib/jackson-jaxrs-base-2.9.9.jar /opt/lagou/servers/atlas-1.2.0/hook/hive/atlas-hive-plugin-impl/jackson-jaxrs-base-2.9.9.jar [root@Linux122 lib]# ln -s /opt/lagou/servers/atlas-1.2.0/server/webapp/atlas/WEB-INF/lib/jackson-jaxrs-json-provider-2.9.9.jar /opt/lagou/servers/atlas-1.2.0/hook/hive/atlas-hive-plugin-impl/jackson-jaxrs-json-provider-2.9.9.jar [root@Linux122 lib]# ln -s /opt/lagou/servers/atlas-1.2.0/server/webapp/atlas/WEB-INF/lib/jackson-module-jaxb-annotations-2.9.9.jar /opt/lagou/servers/atlas-1.2.0/hook/hive/atlas-hive-plugin-impl/jackson-module-jaxb-annotations-2.9.9.jar [root@Linux122 lib]# cd /opt/lagou/servers/atlas-1.2.0/hook/hive/atlas-hive-plugin-impl/ [root@Linux122 atlas-hive-plugin-impl]# ls atlas-client-common-1.2.0.jar jackson-core-2.9.9.jar atlas-client-v1-1.2.0.jar jackson-databind-2.9.9.jar atlas-client-v2-1.2.0.jar jackson-jaxrs-base-2.9.9.jar atlas-common-1.2.0.jar jackson-jaxrs-json-provider-2.9.9.jar atlas-intg-1.2.0.jar jackson-module-jaxb-annotations-2.9.9.jar atlas-notification-1.2.0.jar jersey-json-1.19.jar commons-configuration-1.10.jar jersey-multipart-1.19.jar hbase-common-1.1.2.jar jsr311-api-1.1.jar hbase-server-1.1.2.jar kafka_2.11-1.0.0.jar hdfs-model-1.2.0.jar kafka-clients-1.0.0.jar hive-bridge-1.2.0.jar scala-library-2.11.12.jar jackson-annotations-2.9.9.jar [root@Linux122 atlas-hive-plugin-impl]# ll total 21996 -rw-r--r--. 1 root root 37040 Feb 1 16:36 atlas-client-common-1.2.0.jar -rw-r--r--. 1 root root 41829 Feb 1 16:36 atlas-client-v1-1.2.0.jar -rw-r--r--. 1 root root 19534 Feb 1 16:36 atlas-client-v2-1.2.0.jar -rw-r--r--. 1 root root 70743 Feb 1 16:36 atlas-common-1.2.0.jar -rw-r--r--. 1 root root 484994 Feb 1 16:36 atlas-intg-1.2.0.jar -rw-r--r--. 1 root root 59523 Feb 1 16:36 atlas-notification-1.2.0.jar -rw-r--r--. 1 root root 362679 Jan 30 14:52 commons-configuration-1.10.jar -rw-r--r--. 1 root root 530078 Jan 30 15:17 hbase-common-1.1.2.jar -rw-r--r--. 1 root root 3978343 Jan 30 15:18 hbase-server-1.1.2.jar -rw-r--r--. 1 root root 10869 Feb 1 16:42 hdfs-model-1.2.0.jar -rw-r--r--. 1 root root 80919 Feb 1 16:43 hive-bridge-1.2.0.jar -rw-r--r--. 1 root root 66897 Jan 30 14:53 jackson-annotations-2.9.9.jar -rw-r--r--. 1 root root 325632 Jan 30 14:53 jackson-core-2.9.9.jar -rw-r--r--. 1 root root 1348108 Jan 30 14:53 jackson-databind-2.9.9.jar lrwxrwxrwx. 1 root root 91 Feb 1 18:34 jackson-jaxrs-base-2.9.9.jar -> /opt/lagou/servers/atlas-1.2.0/server/webapp/atlas/WEB-INF/lib/jackson-jaxrs-base-2.9.9.jar lrwxrwxrwx. 1 root root 100 Feb 1 18:35 jackson-jaxrs-json-provider-2.9.9.jar -> /opt/lagou/servers/atlas-1.2.0/server/webapp/atlas/WEB-INF/lib/jackson-jaxrs-json-provider-2.9.9.jar lrwxrwxrwx. 1 root root 104 Feb 1 18:37 jackson-module-jaxb-annotations-2.9.9.jar -> /opt/lagou/servers/atlas-1.2.0/server/webapp/atlas/WEB-INF/lib/jackson-module-jaxb-annotations-2.9.9.jar -rw-r--r--. 1 root root 165345 Jan 30 19:08 jersey-json-1.19.jar -rw-r--r--. 1 root root 53275 Jan 30 14:53 jersey-multipart-1.19.jar -rw-r--r--. 1 root root 45927 Jan 30 19:08 jsr311-api-1.1.jar -rw-r--r--. 1 root root 7452292 Jan 30 14:53 kafka_2.11-1.0.0.jar -rw-r--r--. 1 root root 1591338 Jan 30 14:53 kafka-clients-1.0.0.jar -rw-r--r--. 1 root root 5749423 Jan 30 19:08 scala-library-2.11.12.jar [root@Linux122 atlas-hive-plugin-impl]#3.修改hive配置
/opt/lagou/servers/hive-2.3.7/conf/hive-site.xml
增加配置
hive.exec.post.hooks org.apache.atlas.hive.hook.HiveHook
/opt/lagou/servers/hive-2.3.7/conf/hive-env.sh
新增配置,告诉jar包地址
export HIVE_AUX_JARS_PATH=/opt/lagou/servers/atlas-1.2.0/hook/hive4.批量导入hive元数据
执行以下脚本
/opt/lagou/servers/atlas-1.2.0/bin/import-hive.sh错误一
[root@Linux122 bin]# import-hive.sh Using Hive configuration directory [/opt/lagou/servers/hive-2.3.7/conf] Log file for import is /opt/lagou/servers/atlas-1.2.0/logs/import-hive.log 2022-02-01T18:57:51,339 INFO [main] org.apache.atlas.ApplicationProperties - Looking for atlas-application.properties in classpath 2022-02-01T18:57:51,350 INFO [main] org.apache.atlas.ApplicationProperties - Loading atlas-application.properties from file:/opt/lagou/servers/hive-2.3.7/conf/atlas-application.properties 2022-02-01T18:57:51,608 INFO [main] org.apache.atlas.ApplicationProperties - Property (set to default) atlas.graph.cache.db-cache = true 2022-02-01T18:57:51,608 INFO [main] org.apache.atlas.ApplicationProperties - Property (set to default) atlas.graph.cache.db-cache-clean-wait = 20 2022-02-01T18:57:51,608 INFO [main] org.apache.atlas.ApplicationProperties - Property (set to default) atlas.graph.cache.db-cache-size = 0.5 2022-02-01T18:57:51,608 INFO [main] org.apache.atlas.ApplicationProperties - Property (set to default) atlas.graph.cache.tx-cache-size = 15000 2022-02-01T18:57:51,608 INFO [main] org.apache.atlas.ApplicationProperties - Property (set to default) atlas.graph.cache.tx-dirty-size = 120 Enter username for atlas :- admin Enter password for atlas :- Exception in thread "main" java.lang.NoClassDefFoundError: com/sun/jersey/core/util/FeaturesAndProperties at java.lang.ClassLoader.defineClass1(Native Method) at java.lang.ClassLoader.defineClass(ClassLoader.java:756) at java.security.SecureClassLoader.defineClass(SecureClassLoader.java:142) at java.net.URLClassLoader.defineClass(URLClassLoader.java:468) at java.net.URLClassLoader.access0(URLClassLoader.java:74) at java.net.URLClassLoader.run(URLClassLoader.java:369) at java.net.URLClassLoader.run(URLClassLoader.java:363) at java.security.AccessController.doPrivileged(Native Method) at java.net.URLClassLoader.findClass(URLClassLoader.java:362) at java.lang.ClassLoader.loadClass(ClassLoader.java:418) at sun.misc.Launcher$AppClassLoader.loadClass(Launcher.java:355) at java.lang.ClassLoader.loadClass(ClassLoader.java:351) at org.apache.atlas.hive.bridge.HivemetaStoreBridge.main(HivemetaStoreBridge.java:131) Caused by: java.lang.ClassNotFoundException: com.sun.jersey.core.util.FeaturesAndProperties at java.net.URLClassLoader.findClass(URLClassLoader.java:382) at java.lang.ClassLoader.loadClass(ClassLoader.java:418) at sun.misc.Launcher$AppClassLoader.loadClass(Launcher.java:355) at java.lang.ClassLoader.loadClass(ClassLoader.java:351) ... 13 more Failed to import Hive meta Data!!! [root@Linux122 bin]#
注意启动服务前将hdfs-yarn metastore 服务起来保证hive能正常使用
错误二Exception in thread "main" java.lang.NoClassDefFoundError: com/sun/jersey/core/util/FeaturesAndProperties at java.lang.ClassLoader.defineClass1(Native Method) at java.lang.ClassLoader.defineClass(ClassLoader.java:756) at java.security.SecureClassLoader.defineClass(SecureClassLoader.java:142) at java.net.URLClassLoader.defineClass(URLClassLoader.java:468) at java.net.URLClassLoader.access0(URLClassLoader.java:74) at java.net.URLClassLoader.run(URLClassLoader.java:369) at java.net.URLClassLoader.run(URLClassLoader.java:363) at java.security.AccessController.doPrivileged(Native Method) at java.net.URLClassLoader.findClass(URLClassLoader.java:362) at java.lang.ClassLoader.loadClass(ClassLoader.java:418) at sun.misc.Launcher$AppClassLoader.loadClass(Launcher.java:355) at java.lang.ClassLoader.loadClass(ClassLoader.java:351) at org.apache.atlas.hive.bridge.HivemetaStoreBridge.main(HivemetaStoreBridge.java:131) Caused by: java.lang.ClassNotFoundException: com.sun.jersey.core.util.FeaturesAndProperties at java.net.URLClassLoader.findClass(URLClassLoader.java:382) at java.lang.ClassLoader.loadClass(ClassLoader.java:418) at sun.misc.Launcher$AppClassLoader.loadClass(Launcher.java:355) at java.lang.ClassLoader.loadClass(ClassLoader.java:351) ... 13 more Failed to import Hive meta Data!!!
通过网上查找说缺少以下jar包
jersey-core-1.9.jar jersey-server-1.9.jar jersey-client-1.9.jar
我们在上边配置引入相关jar包时候发现有以上包
[root@Linux122 lib]# ls jersey* jersey-client-1.19.jar jersey-json-1.9.jar jersey-server-1.19.jar jersey-spring-1.19.4.jar jersey-core-1.19.jar jersey-multipart-1.19.jar jersey-servlet-1.19.jar [root@Linux122 lib]# pwd /opt/lagou/servers/atlas-1.2.0/server/webapp/atlas/WEB-INF/lib [root@Linux122 lib]# ln -s /opt/lagou/servers/atlas-1.2.0/server/webapp/atlas/WEB-INF/lib/jersey-core-1.19.jar /opt/lagou/servers/atlas-1.2.0/hook/hive/atlas-hive-plugin-impl/jersey-core-1.19.jar [root@Linux122 lib]# ln -s /opt/lagou/servers/atlas-1.2.0/server/webapp/atlas/WEB-INF/lib/jersey-server-1.19.jar /opt/lagou/servers/atlas-1.2.0/hook/hive/atlas-hive-plugin-impl/jersey-server-1.19.jar [root@Linux122 lib]# ln -s /opt/lagou/servers/atlas-1.2.0/server/webapp/atlas/WEB-INF/lib/jersey-client-1.19.jar /opt/lagou/servers/atlas-1.2.0/hook/hive/atlas-hive-plugin-impl/jersey-client-1.19.jar错误三
Exception in thread "main" java.lang.NoClassDefFoundError: org/apache/hadoop/security/authentication/client/ConnectionConfigurator at org.apache.atlas.AtlasbaseClient.getClient(AtlasbaseClient.java:287) at org.apache.atlas.AtlasbaseClient.initializeState(AtlasbaseClient.java:453) at org.apache.atlas.AtlasbaseClient.initializeState(AtlasbaseClient.java:448) at org.apache.atlas.AtlasbaseClient.(AtlasbaseClient.java:132) at org.apache.atlas.AtlasClientV2. (AtlasClientV2.java:82) at org.apache.atlas.hive.bridge.HivemetaStoreBridge.main(HivemetaStoreBridge.java:131) Caused by: java.lang.ClassNotFoundException: org.apache.hadoop.security.authentication.client.ConnectionConfigurator at java.net.URLClassLoader.findClass(URLClassLoader.java:382) at java.lang.ClassLoader.loadClass(ClassLoader.java:418) at sun.misc.Launcher$AppClassLoader.loadClass(Launcher.java:355) at java.lang.ClassLoader.loadClass(ClassLoader.java:351) ... 6 more Failed to import Hive meta Data!!!
继续寻找jar包
ln -s /opt/lagou/servers/atlas-1.2.0/server/webapp/atlas/WEB-INF/lib/hadoop-auth-2.9.2.jar /opt/lagou/servers/atlas-1.2.0/hook/hive/atlas-hive-plugin-impl/hadoop-auth-2.9.2.jar错误四
Exception in thread "main" java.lang.NoClassDefFoundError: org/apache/hadoop/conf/Configuration at java.lang.ClassLoader.defineClass1(Native Method) at java.lang.ClassLoader.defineClass(ClassLoader.java:756) at java.security.SecureClassLoader.defineClass(SecureClassLoader.java:142) at java.net.URLClassLoader.defineClass(URLClassLoader.java:468) at java.net.URLClassLoader.access0(URLClassLoader.java:74) at java.net.URLClassLoader.run(URLClassLoader.java:369) at java.net.URLClassLoader.run(URLClassLoader.java:363) at java.security.AccessController.doPrivileged(Native Method) at java.net.URLClassLoader.findClass(URLClassLoader.java:362) at java.lang.ClassLoader.loadClass(ClassLoader.java:418) at sun.misc.Launcher$AppClassLoader.loadClass(Launcher.java:355) at java.lang.ClassLoader.loadClass(ClassLoader.java:351) at org.apache.atlas.hive.bridge.HivemetaStoreBridge.main(HivemetaStoreBridge.java:138) Caused by: java.lang.ClassNotFoundException: org.apache.hadoop.conf.Configuration at java.net.URLClassLoader.findClass(URLClassLoader.java:382) at java.lang.ClassLoader.loadClass(ClassLoader.java:418) at sun.misc.Launcher$AppClassLoader.loadClass(Launcher.java:355) at java.lang.ClassLoader.loadClass(ClassLoader.java:351) ... 13 more Failed to import Hive meta Data!!!
看样子缺少hadoop依赖,我将他们都软连接了
[root@Linux122 lib]# ln -s /opt/lagou/servers/atlas-1.2.0/server/webapp/atlas/WEB-INF/lib/hadoop-annotations-2.9.2.jar /opt/lagou/servers/atlas-1.2.0/hook/hive/atlas-hive-plugin-impl/hadoop-annotations-2.9.2.jar [root@Linux122 lib]# ln -s /opt/lagou/servers/atlas-1.2.0/server/webapp/atlas/WEB-INF/lib/hadoop-common-2.9.2.jar /opt/lagou/servers/atlas-1.2.0/hook/hive/atlas-hive-plugin-impl/hadoop-common-2.9.2.jar [root@Linux122 lib]# ln -s /opt/lagou/servers/atlas-1.2.0/server/webapp/atlas/WEB-INF/lib/hadoop-hdfs-2.9.2.jar /opt/lagou/servers/atlas-1.2.0/hook/hive/atlas-hive-plugin-impl/hadoop-hdfs-2.9.2.jar [root@Linux122 lib]# ln -s /opt/lagou/servers/atlas-1.2.0/server/webapp/atlas/WEB-INF/lib/hadoop-hdfs-client-2.9.2.jar /opt/lagou/servers/atlas-1.2.0/hook/hive/atlas-hive-plugin-impl/hadoop-hdfs-client-2.9.2.jar错误五
Exception in thread "main" java.lang.NoClassDefFoundError: com/ctc/wstx/io/InputBootstrapper at org.apache.atlas.hive.bridge.HivemetaStoreBridge.main(HivemetaStoreBridge.java:138) Caused by: java.lang.ClassNotFoundException: com.ctc.wstx.io.InputBootstrapper at java.net.URLClassLoader.findClass(URLClassLoader.java:382) at java.lang.ClassLoader.loadClass(ClassLoader.java:418) at sun.misc.Launcher$AppClassLoader.loadClass(Launcher.java:355) at java.lang.ClassLoader.loadClass(ClassLoader.java:351) ... 1 more Failed to import Hive meta Data!!!
缺少如下jar包
[root@Linux122 lib]# ln -s /opt/lagou/servers/atlas-1.2.0/server/webapp/atlas/WEB-INF/lib/woodstox-core-5.0.3.jar /opt/lagou/servers/atlas-1.2.0/hook/hive/atlas-hive-plugin-impl/woodstox-core-5.0.3.jar错误六
Exception in thread "main" java.lang.NoClassDefFoundError: org/codehaus/stax2/XMLInputFactory2 at java.lang.ClassLoader.defineClass1(Native Method) at java.lang.ClassLoader.defineClass(ClassLoader.java:756) at java.security.SecureClassLoader.defineClass(SecureClassLoader.java:142) at java.net.URLClassLoader.defineClass(URLClassLoader.java:468) at java.net.URLClassLoader.access$100(URLClassLoader.java:74) at java.net.URLClassLoader$1.run(URLClassLoader.java:369) at java.net.URLClassLoader$1.run(URLClassLoader.java:363) at java.security.AccessController.doPrivileged(Native Method) at java.net.URLClassLoader.findClass(URLClassLoader.java:362) at java.lang.ClassLoader.loadClass(ClassLoader.java:418) at sun.misc.Launcher$AppClassLoader.loadClass(Launcher.java:355) at java.lang.ClassLoader.loadClass(ClassLoader.java:351) at org.apache.hadoop.conf.Configuration.(Configuration.java:312) at org.apache.atlas.hive.bridge.HivemetaStoreBridge.main(HivemetaStoreBridge.java:138) Caused by: java.lang.ClassNotFoundException: org.codehaus.stax2.XMLInputFactory2 at java.net.URLClassLoader.findClass(URLClassLoader.java:382) at java.lang.ClassLoader.loadClass(ClassLoader.java:418) at sun.misc.Launcher$AppClassLoader.loadClass(Launcher.java:355) at java.lang.ClassLoader.loadClass(ClassLoader.java:351) ... 14 more Failed to import Hive meta Data!!! [root@Linux122 bin]#
继续
[root@Linux122 lib]# ln -s /opt/lagou/servers/atlas-1.2.0/server/webapp/atlas/WEB-INF/lib/stax2-api-3.1.4.jar /opt/lagou/servers/atlas-1.2.0/hook/hive/atlas-hive-plugin-impl/stax2-api-3.1.4.jar错误六
2022-02-02T10:58:54,673 INFO [main] hive.metastore - Connected to metastore. 2022-02-02T10:58:54,996 INFO [main] org.apache.atlas.hive.bridge.HivemetaStoreBridge - importing Hive metadata 2022-02-02T10:58:55,022 INFO [main] org.apache.atlas.hive.bridge.HivemetaStoreBridge - Found 11 databases 2022-02-02T10:58:55,339 ERROR [main] org.apache.atlas.hive.bridge.HivemetaStoreBridge - import failed com.sun.jersey.api.client.ClientHandlerException: java.net.ConnectException: Connection refused (Connection refused) at com.sun.jersey.client.urlconnection.URLConnectionClientHandler.handle(URLConnectionClientHandler.java:155) ~[jersey-client-1.19.jar:1.19] at com.sun.jersey.api.client.filter.HTTPBasicAuthFilter.handle(HTTPBasicAuthFilter.java:105) ~[jersey-client-1.19.jar:1.19] at com.sun.jersey.api.client.Client.handle(Client.java:652) ~[jersey-client-1.19.jar:1.19] at com.sun.jersey.api.client.WebResource.handle(WebResource.java:682) ~[jersey-client-1.19.jar:1.19] at com.sun.jersey.api.client.WebResource.access$200(WebResource.java:74) ~[jersey-client-1.19.jar:1.19] at com.sun.jersey.api.client.WebResource$Builder.method(WebResource.java:634) ~[jersey-client-1.19.jar:1.19] at org.apache.atlas.AtlasbaseClient.callAPIWithResource(AtlasbaseClient.java:382) ~[atlas-client-common-1.2.0.jar:1.2.0] at org.apache.atlas.AtlasbaseClient.callAPIWithResource(AtlasbaseClient.java:353) ~[atlas-client-common-1.2.0.jar:1.2.0] at org.apache.atlas.AtlasbaseClient.callAPI(AtlasbaseClient.java:246) ~[atlas-client-common-1.2.0.jar:1.2.0] at org.apache.atlas.AtlasClientV2.getEntityByAttribute(AtlasClientV2.java:285) ~[atlas-client-v2-1.2.0.jar:1.2.0] at org.apache.atlas.AtlasClientV2.getEntityByAttribute(AtlasClientV2.java:276) ~[atlas-client-v2-1.2.0.jar:1.2.0] at org.apache.atlas.hive.bridge.HivemetaStoreBridge.findEntity(HivemetaStoreBridge.java:780) ~[hive-bridge-1.2.0.jar:1.2.0] at org.apache.atlas.hive.bridge.HivemetaStoreBridge.findDatabase(HivemetaStoreBridge.java:745) ~[hive-bridge-1.2.0.jar:1.2.0] at org.apache.atlas.hive.bridge.HivemetaStoreBridge.registerDatabase(HivemetaStoreBridge.java:399) ~[hive-bridge-1.2.0.jar:1.2.0] at org.apache.atlas.hive.bridge.HivemetaStoreBridge.importDatabases(HivemetaStoreBridge.java:281) ~[hive-bridge-1.2.0.jar:1.2.0] at org.apache.atlas.hive.bridge.HivemetaStoreBridge.importHivemetadata(HivemetaStoreBridge.java:251) ~[hive-bridge-1.2.0.jar:1.2.0] at org.apache.atlas.hive.bridge.HivemetaStoreBridge.main(HivemetaStoreBridge.java:168) [hive-bridge-1.2.0.jar:1.2.0] Caused by: java.net.ConnectException: Connection refused (Connection refused) at java.net.PlainSocketImpl.socketConnect(Native Method) ~[?:1.8.0_301] at java.net.AbstractPlainSocketImpl.doConnect(AbstractPlainSocketImpl.java:476) ~[?:1.8.0_301] at java.net.AbstractPlainSocketImpl.connectToAddress(AbstractPlainSocketImpl.java:218) ~[?:1.8.0_301] at java.net.AbstractPlainSocketImpl.connect(AbstractPlainSocketImpl.java:200) ~[?:1.8.0_301] at java.net.SocksSocketImpl.connect(SocksSocketImpl.java:394) ~[?:1.8.0_301] at java.net.Socket.connect(Socket.java:606) ~[?:1.8.0_301] at sun.net.NetworkClient.doConnect(NetworkClient.java:175) ~[?:1.8.0_301] at sun.net.www.http.HttpClient.openServer(HttpClient.java:463) ~[?:1.8.0_301] at sun.net.www.http.HttpClient.openServer(HttpClient.java:558) ~[?:1.8.0_301] at sun.net.www.http.HttpClient.(HttpClient.java:242) ~[?:1.8.0_301] at sun.net.www.http.HttpClient.New(HttpClient.java:339) ~[?:1.8.0_301] at sun.net.www.http.HttpClient.New(HttpClient.java:357) ~[?:1.8.0_301] at sun.net.www.protocol.http.HttpURLConnection.getNewHttpClient(HttpURLConnection.java:1226) ~[?:1.8.0_301] at sun.net.www.protocol.http.HttpURLConnection.plainConnect0(HttpURLConnection.java:1162) ~[?:1.8.0_301] at sun.net.www.protocol.http.HttpURLConnection.plainConnect(HttpURLConnection.java:1056) ~[?:1.8.0_301] at sun.net.www.protocol.http.HttpURLConnection.connect(HttpURLConnection.java:990) ~[?:1.8.0_301] at sun.net.www.protocol.http.HttpURLConnection.getInputStream0(HttpURLConnection.java:1570) ~[?:1.8.0_301] at sun.net.www.protocol.http.HttpURLConnection.getInputStream(HttpURLConnection.java:1498) ~[?:1.8.0_301] at java.net.HttpURLConnection.getResponseCode(HttpURLConnection.java:480) ~[?:1.8.0_301] at com.sun.jersey.client.urlconnection.URLConnectionClientHandler._invoke(URLConnectionClientHandler.java:253) ~[jersey-client-1.19.jar:1.19] at com.sun.jersey.client.urlconnection.URLConnectionClientHandler.handle(URLConnectionClientHandler.java:153) ~[jersey-client-1.19.jar:1.19] ... 16 more Failed to import Hive meta Data!!!
通过查资料看官方文档
重新启动Atlas
atlas_stop.py
Running Apache Atlas with Local Apache Hbase & Apache Solr
To run Apache Atlas with local Apache Hbase & Apache Solr instances that are started/stopped along with Atlas start/stop, run following commands:
export MANAGE_LOCAL_Hbase=true export MANAGE_LOCAL_SOLR=true bin/atlas_start.py
检查服务启动状态
[root@Linux122 bin]# curl -u admin:admin http://localhost:21000/api/atlas/admin/version
{"Description":"metadata Management and Data Governance Platform over Hadoop","Revision":"release","Version":"1.2.0","Name":"apache-atlas"}
错误七
2022-02-02T13:42:06,735 ERROR [main] org.apache.atlas.hive.bridge.HivemetaStoreBridge - import failed com.sun.jersey.api.client.ClientHandlerException: java.net.SocketTimeoutException: Read timed out at com.sun.jersey.client.urlconnection.URLConnectionClientHandler.handle(URLConnectionClientHandler.java:155) ~[jersey-client-1.19.jar:1.19] at com.sun.jersey.api.client.filter.HTTPBasicAuthFilter.handle(HTTPBasicAuthFilter.java:105) ~[jersey-client-1.19.jar:1.19] at com.sun.jersey.api.client.Client.handle(Client.java:652) ~[jersey-client-1.19.jar:1.19] at com.sun.jersey.api.client.WebResource.handle(WebResource.java:682) ~[jersey-client-1.19.jar:1.19] at com.sun.jersey.api.client.WebResource.access0(WebResource.java:74) ~[jersey-client-1.19.jar:1.19] at com.sun.jersey.api.client.WebResource$Builder.method(WebResource.java:634) ~[jersey-client-1.19.jar:1.19] at org.apache.atlas.AtlasbaseClient.callAPIWithResource(AtlasbaseClient.java:382) ~[atlas-client-common-1.2.0.jar:1.2.0] at org.apache.atlas.AtlasbaseClient.callAPIWithResource(AtlasbaseClient.java:353) ~[atlas-client-common-1.2.0.jar:1.2.0] at org.apache.atlas.AtlasbaseClient.callAPI(AtlasbaseClient.java:246) ~[atlas-client-common-1.2.0.jar:1.2.0] at org.apache.atlas.AtlasClientV2.getEntityByAttribute(AtlasClientV2.java:285) ~[atlas-client-v2-1.2.0.jar:1.2.0] at org.apache.atlas.AtlasClientV2.getEntityByAttribute(AtlasClientV2.java:276) ~[atlas-client-v2-1.2.0.jar:1.2.0] at org.apache.atlas.hive.bridge.HivemetaStoreBridge.findEntity(HivemetaStoreBridge.java:780) ~[hive-bridge-1.2.0.jar:1.2.0] at org.apache.atlas.hive.bridge.HivemetaStoreBridge.findDatabase(HivemetaStoreBridge.java:745) ~[hive-bridge-1.2.0.jar:1.2.0] at org.apache.atlas.hive.bridge.HivemetaStoreBridge.registerDatabase(HivemetaStoreBridge.java:399) ~[hive-bridge-1.2.0.jar:1.2.0] at org.apache.atlas.hive.bridge.HivemetaStoreBridge.importDatabases(HivemetaStoreBridge.java:281) ~[hive-bridge-1.2.0.jar:1.2.0] at org.apache.atlas.hive.bridge.HivemetaStoreBridge.importHivemetadata(HivemetaStoreBridge.java:251) ~[hive-bridge-1.2.0.jar:1.2.0] at org.apache.atlas.hive.bridge.HivemetaStoreBridge.main(HivemetaStoreBridge.java:168) [hive-bridge-1.2.0.jar:1.2.0] Caused by: java.net.SocketTimeoutException: Read timed out at java.net.SocketInputStream.socketRead0(Native Method) ~[?:1.8.0_301] at java.net.SocketInputStream.socketRead(SocketInputStream.java:116) ~[?:1.8.0_301] at java.net.SocketInputStream.read(SocketInputStream.java:171) ~[?:1.8.0_301] at java.net.SocketInputStream.read(SocketInputStream.java:141) ~[?:1.8.0_301] at java.io.BufferedInputStream.fill(BufferedInputStream.java:246) ~[?:1.8.0_301] at java.io.BufferedInputStream.read1(BufferedInputStream.java:286) ~[?:1.8.0_301] at java.io.BufferedInputStream.read(BufferedInputStream.java:345) ~[?:1.8.0_301] at sun.net.www.http.HttpClient.parseHTTPHeader(HttpClient.java:735) ~[?:1.8.0_301] at sun.net.www.http.HttpClient.parseHTTP(HttpClient.java:678) ~[?:1.8.0_301] at sun.net.www.protocol.http.HttpURLConnection.getInputStream0(HttpURLConnection.java:1593) ~[?:1.8.0_301] at sun.net.www.protocol.http.HttpURLConnection.getInputStream(HttpURLConnection.java:1498) ~[?:1.8.0_301] at java.net.HttpURLConnection.getResponseCode(HttpURLConnection.java:480) ~[?:1.8.0_301] at com.sun.jersey.client.urlconnection.URLConnectionClientHandler._invoke(URLConnectionClientHandler.java:253) ~[jersey-client-1.19.jar:1.19] at com.sun.jersey.client.urlconnection.URLConnectionClientHandler.handle(URLConnectionClientHandler.java:153) ~[jersey-client-1.19.jar:1.19] ... 16 more Failed to import Hive meta Data!!!
通过查资料发现是需要配置超时时间
/opt/lagou/servers/atlas-1.2.0/conf/atlas-application.properties 增加如下配置 atlas.hook.hive.keepAliveTime=1000000
重新导入元数据
[root@Linux122 bin]# import-hive.sh Using Hive configuration directory [/opt/lagou/servers/hive-2.3.7/conf] Log file for import is /opt/lagou/servers/atlas-1.2.0/logs/import-hive.log 2022-02-02T14:30:48,137 INFO [main] org.apache.atlas.ApplicationProperties - Looking for atlas-application.properties in classpath 2022-02-02T14:30:48,144 INFO [main] org.apache.atlas.ApplicationProperties - Loading atlas-application.properties from file:/opt/lagou/servers/hive-2.3.7/conf/atlas-application.properties 2022-02-02T14:30:48,317 INFO [main] org.apache.atlas.ApplicationProperties - Property (set to default) atlas.graph.cache.db-cache = true 2022-02-02T14:30:48,317 INFO [main] org.apache.atlas.ApplicationProperties - Property (set to default) atlas.graph.cache.db-cache-clean-wait = 20 2022-02-02T14:30:48,317 INFO [main] org.apache.atlas.ApplicationProperties - Property (set to default) atlas.graph.cache.db-cache-size = 0.5 2022-02-02T14:30:48,317 INFO [main] org.apache.atlas.ApplicationProperties - Property (set to default) atlas.graph.cache.tx-cache-size = 15000 2022-02-02T14:30:48,318 INFO [main] org.apache.atlas.ApplicationProperties - Property (set to default) atlas.graph.cache.tx-dirty-size = 120 Enter username for atlas :- admin Enter password for atlas :- 2022-02-02T14:30:55,028 INFO [main] org.apache.atlas.AtlasbaseClient - Client has only one service URL, will use that for all actions: http://localhost:21000 2022-02-02T14:30:55,218 INFO [main] org.apache.hadoop.hive.conf.HiveConf - Found configuration file file:/opt/lagou/servers/hive-2.3.7/conf/hive-site.xml 2022-02-02T14:30:57,861 WARN [main] org.apache.hadoop.util.NativeCodeLoader - Unable to load native-hadoop library for your platform... using builtin-java classes where applicable 2022-02-02T14:30:58,003 INFO [main] hive.metastore - Trying to connect to metastore with URI thrift://Linux121:9083 2022-02-02T14:30:58,108 INFO [main] hive.metastore - Opened a connection to metastore, current connections: 1 2022-02-02T14:30:58,240 INFO [main] hive.metastore - Connected to metastore. 2022-02-02T14:30:58,801 INFO [main] org.apache.atlas.hive.bridge.HivemetaStoreBridge - importing Hive metadata 2022-02-02T14:30:58,853 INFO [main] org.apache.atlas.hive.bridge.HivemetaStoreBridge - Found 11 databases 2022-02-02T14:31:00,430 INFO [main] org.apache.atlas.AtlasbaseClient - method=GET path=api/atlas/v2/entity/uniqueAttribute/type/ contentType=application/json; charset=UTF-8 accept=application/json status=200 2022-02-02T14:31:01,109 INFO [main] org.apache.atlas.hive.bridge.HivemetaStoreBridge - Database ads is already registered - id=897fe37d-7eaa-499f-8843-690003f22e9b. Updating it. 2022-02-02T14:31:01,449 INFO [main] org.apache.atlas.AtlasbaseClient - method=POST path=api/atlas/v2/entity/ contentType=application/json; charset=UTF-8 accept=application/json status=200 2022-02-02T14:31:01,465 INFO [main] org.apache.atlas.hive.bridge.HivemetaStoreBridge - Updated hive_db entity: name=ads@primary, guid=897fe37d-7eaa-499f-8843-690003f22e9b 2022-02-02T14:31:02,280 INFO [main] org.apache.atlas.hive.bridge.HivemetaStoreBridge - Found 12 tables to import in database ads 2022-02-02T14:31:03,605 INFO [main] org.apache.atlas.AtlasbaseClient - method=GET path=api/atlas/v2/entity/uniqueAttribute/type/ contentType=application/json; charset=UTF-8 accept=application/json status=200 2022-02-02T14:31:03,607 INFO [main] org.apache.atlas.hive.bridge.HivemetaStoreBridge - Table ads.ads_active_member_frequenty_3of_recent6_cnt is already registered with id addb017c-5389-44c1-9b39-afacf92208d0. Updating entity. 2022-02-02T14:31:05,365 INFO [main] org.apache.atlas.AtlasbaseClient - method=POST path=api/atlas/v2/entity/ contentType=application/json; charset=UTF-8 accept=application/json status=200 2022-02-02T14:31:05,366 INFO [main] org.apache.atlas.hive.bridge.HivemetaStoreBridge - Updated hive_table entity: name=ads.ads_active_member_frequenty_3of_recent6_cnt@primary, guid=addb017c-5389-44c1-9b39-afacf92208d0 2022-02-02T14:31:05,798 INFO [main] org.apache.atlas.AtlasbaseClient - method=GET path=api/atlas/v2/entity/uniqueAttribute/type/ contentType=application/json; charset=UTF-8 accept=application/json status=200 2022-02-02T14:31:05,801 INFO [main] org.apache.atlas.hive.bridge.HivemetaStoreBridge - Table ads.ads_ad_show is already registered with id ccd6ab90-fed1-4530-b03d-0b76e56950f9. Updating entity. 2022-02-02T14:31:06,912 INFO [main] org.apache.atlas.AtlasbaseClient - method=POST path=api/atlas/v2/entity/ contentType=application/json; charset=UTF-8 accept=application/json status=200 2022-02-02T14:31:06,925 INFO [main] org.apache.atlas.hive.bridge.HivemetaStoreBridge - Updated hive_table entity: name=ads.ads_ad_show@primary, guid=ccd6ab90-fed1-4530-b03d-0b76e56950f9 2022-02-02T14:31:07,303 INFO [main] org.apache.atlas.AtlasbaseClient - method=GET path=api/atlas/v2/entity/uniqueAttribute/type/ contentType=application/json; charset=UTF-8 accept=application/json status=200 2022-02-02T14:31:07,309 INFO [main] org.apache.atlas.hive.bridge.HivemetaStoreBridge - Table ads.ads_ad_show_place is already registered with id 7ea4a597-2a7d-412d-8b85-704b41d3f183. Updating entity. 2022-02-02T14:31:07,861 INFO [main] org.apache.atlas.AtlasbaseClient - method=POST path=api/atlas/v2/entity/ contentType=application/json; charset=UTF-8 accept=application/json status=200 2022-02-02T14:31:07,862 INFO [main] org.apache.atlas.hive.bridge.HivemetaStoreBridge - Updated hive_table entity: name=ads.ads_ad_show_place@primary, guid=7ea4a597-2a7d-412d-8b85-704b41d3f183 2022-02-02T14:31:08,293 INFO [main] org.apache.atlas.AtlasbaseClient - method=GET path=api/atlas/v2/entity/uniqueAttribute/type/ contentType=application/json; charset=UTF-8 accept=application/json status=200 2022-02-02T14:31:08,295 INFO [main] org.apache.atlas.hive.bridge.HivemetaStoreBridge - Table ads.ads_ad_show_place_window is already registered with id 79bda62d-3132-4acc-8729-95e3b7426dda. Updating entity. 2022-02-02T14:31:09,438 INFO [main] org.apache.atlas.AtlasbaseClient - method=POST path=api/atlas/v2/entity/ contentType=application/json; charset=UTF-8 accept=application/json status=200 2022-02-02T14:31:09,443 INFO [main] org.apache.atlas.hive.bridge.HivemetaStoreBridge - Updated hive_table entity: name=ads.ads_ad_show_place_window@primary, guid=79bda62d-3132-4acc-8729-95e3b7426dda 2022-02-02T14:31:09,833 INFO [main] org.apache.atlas.AtlasbaseClient - method=GET path=api/atlas/v2/entity/uniqueAttribute/type/ contentType=application/json; charset=UTF-8 accept=application/json status=200 2022-02-02T14:31:09,837 INFO [main] org.apache.atlas.hive.bridge.HivemetaStoreBridge - Table ads.ads_ad_show_rate is already registered with id e67b2530-d5e4-4da7-88a5-19b60581cf4e. Updating entity. 2022-02-02T14:31:12,648 INFO [main] org.apache.atlas.AtlasbaseClient - method=POST path=api/atlas/v2/entity/ contentType=application/json; charset=UTF-8 accept=application/json status=200 2022-02-02T14:31:12,686 INFO [main] org.apache.atlas.hive.bridge.HivemetaStoreBridge - Updated hive_table entity: name=ads.ads_ad_show_rate@primary, guid=e67b2530-d5e4-4da7-88a5-19b60581cf4e 2022-02-02T14:31:13,124 INFO [main] org.apache.atlas.AtlasbaseClient - method=GET path=api/atlas/v2/entity/uniqueAttribute/type/ contentType=application/json; charset=UTF-8 accept=application/json status=200 2022-02-02T14:31:13,126 INFO [main] org.apache.atlas.hive.bridge.HivemetaStoreBridge - Table ads.ads_member_active_count is already registered with id 50271cdb-74b9-42f3-a537-340d2678b2d9. Updating entity. 2022-02-02T14:31:13,750 INFO [main] org.apache.atlas.AtlasbaseClient - method=POST path=api/atlas/v2/entity/ contentType=application/json; charset=UTF-8 accept=application/json status=200 2022-02-02T14:31:13,751 INFO [main] org.apache.atlas.hive.bridge.HivemetaStoreBridge - Updated hive_table entity: name=ads.ads_member_active_count@primary, guid=50271cdb-74b9-42f3-a537-340d2678b2d9 2022-02-02T14:31:14,050 INFO [main] org.apache.atlas.AtlasbaseClient - method=GET path=api/atlas/v2/entity/uniqueAttribute/type/ contentType=application/json; charset=UTF-8 accept=application/json status=200 2022-02-02T14:31:14,052 INFO [main] org.apache.atlas.hive.bridge.HivemetaStoreBridge - Table ads.ads_member_retention_count is already registered with id dd245b45-f234-46ab-94d2-01a318f83b06. Updating entity. 2022-02-02T14:31:14,500 INFO [main] org.apache.atlas.AtlasbaseClient - method=POST path=api/atlas/v2/entity/ contentType=application/json; charset=UTF-8 accept=application/json status=200 2022-02-02T14:31:14,501 INFO [main] org.apache.atlas.hive.bridge.HivemetaStoreBridge - Updated hive_table entity: name=ads.ads_member_retention_count@primary, guid=dd245b45-f234-46ab-94d2-01a318f83b06 2022-02-02T14:31:14,663 INFO [main] org.apache.atlas.AtlasbaseClient - method=GET path=api/atlas/v2/entity/uniqueAttribute/type/ contentType=application/json; charset=UTF-8 accept=application/json status=200 2022-02-02T14:31:14,664 INFO [main] org.apache.atlas.hive.bridge.HivemetaStoreBridge - Table ads.ads_member_retention_rate is already registered with id 706f5111-f1eb-4c43-b689-ec9824b0a30b. Updating entity. 2022-02-02T14:31:15,882 INFO [main] org.apache.atlas.AtlasbaseClient - method=POST path=api/atlas/v2/entity/ contentType=application/json; charset=UTF-8 accept=application/json status=200 2022-02-02T14:31:15,883 INFO [main] org.apache.atlas.hive.bridge.HivemetaStoreBridge - Updated hive_table entity: name=ads.ads_member_retention_rate@primary, guid=706f5111-f1eb-4c43-b689-ec9824b0a30b 2022-02-02T14:31:16,075 INFO [main] org.apache.atlas.AtlasbaseClient - method=GET path=api/atlas/v2/entity/uniqueAttribute/type/ contentType=application/json; charset=UTF-8 accept=application/json status=200 2022-02-02T14:31:16,076 INFO [main] org.apache.atlas.hive.bridge.HivemetaStoreBridge - Table ads.ads_new_member_cnt is already registered with id ec6ac531-7faf-4f5f-bd3f-e6a27da84702. Updating entity. 2022-02-02T14:31:16,649 INFO [main] org.apache.atlas.AtlasbaseClient - method=POST path=api/atlas/v2/entity/ contentType=application/json; charset=UTF-8 accept=application/json status=200 2022-02-02T14:31:16,650 INFO [main] org.apache.atlas.hive.bridge.HivemetaStoreBridge - Updated hive_table entity: name=ads.ads_new_member_cnt@primary, guid=ec6ac531-7faf-4f5f-bd3f-e6a27da84702 2022-02-02T14:31:16,890 INFO [main] org.apache.atlas.AtlasbaseClient - method=GET path=api/atlas/v2/entity/uniqueAttribute/type/ contentType=application/json; charset=UTF-8 accept=application/json status=200 2022-02-02T14:31:16,892 INFO [main] org.apache.atlas.hive.bridge.HivemetaStoreBridge - Table ads.ads_trade_order_analysis is already registered with id 77f518f8-2cc1-4255-a8f6-8d2d1886e577. Updating entity. 2022-02-02T14:31:17,906 INFO [main] org.apache.atlas.AtlasbaseClient - method=POST path=api/atlas/v2/entity/ contentType=application/json; charset=UTF-8 accept=application/json status=200 2022-02-02T14:31:17,909 INFO [main] org.apache.atlas.hive.bridge.HivemetaStoreBridge - Updated hive_table entity: name=ads.ads_trade_order_analysis@primary, guid=77f518f8-2cc1-4255-a8f6-8d2d1886e577 2022-02-02T14:31:18,040 INFO [main] org.apache.atlas.AtlasbaseClient - method=GET path=api/atlas/v2/entity/uniqueAttribute/type/ contentType=application/json; charset=UTF-8 accept=application/json status=200 .......... 2022-02-02T14:32:05,936 INFO [main] org.apache.atlas.AtlasbaseClient - method=GET path=api/atlas/v2/entity/uniqueAttribute/type/ contentType=application/json; charset=UTF-8 accept=application/json status=404 2022-02-02T14:32:06,040 INFO [main] org.apache.atlas.AtlasbaseClient - method=POST path=api/atlas/v2/entity/ contentType=application/json; charset=UTF-8 accept=application/json status=200 2022-02-02T14:32:06,089 INFO [main] org.apache.atlas.AtlasbaseClient - method=GET path=api/atlas/v2/entity/guid/ contentType=application/json; charset=UTF-8 accept=application/json status=200 2022-02-02T14:32:06,089 INFO [main] org.apache.atlas.hive.bridge.HivemetaStoreBridge - Created hive_db entity: name=tmp@primary, guid=e5334f76-3bb3-4052-a27d-ada5496aaf1b 2022-02-02T14:32:06,101 INFO [main] org.apache.atlas.hive.bridge.HivemetaStoreBridge - No tables to import in database tmp Hive meta Data imported successfully!!!
终于成功了
查看web界面
血缘关系展示
完美应用起来!!!
欢迎分享，转载请注明来源：内存溢出
微信扫一扫
支付宝扫一扫
评论列表（0条）