Re: [SUSPICIOUS] Hello, PNDA team. I have some questions about PNDA.

[Donald Hunter (donaldh)]

Hello Robin,

I have copied your email to the pnda-users group so other community members can read it.

I don’t recognise the specific error, but problems like this are often caused by upstream package updates. I think there have been a couple of new CentOS 7 releases since PNDA 5.0 was released and there is likely to be some impact.

You don’t mention whether you are working with release/5.0 or develop as the git revision. There have been some fixes pushed to develop to resolve upstream changes.

I will need to check my notes for recommendations of how to debug the salt install in more detail.

I just tried to build a mirror and it’s failing because of a missing python3 binary so I’ll look at resolving that, and then see if I can validate an Openstack install.

The correct credentials for the PNDA console are pnda/pnda so if that does not work then it looks like relevant parts of the PNDA install have not completed.

Thanks,

Donald.

On 26 Apr 2019, at 10:09, lu....@zte.com.cn wrote:

Hello, PNDA team.

My name is Robin, and I'm a developer in cloud computing of ZTE Corporation. I'm interested in PNDA and want to know about its function and operating principle. So I try to build a experimental environment following the guide of PNDA at http://secure-web.cisco.com/1PP51Ne62L0b-NZdRerQL39wU-iHkf41Rb6idNO-A6lAx-FnDk6FgplRu7GibliuJ9vt4UTu63MrBJJEq6RsBjisnOsc0O_P0QC9PV5n6zAKepKbiwHsk8YxtpUVPndJHumTn5_21SetfhRH0vJBkf9xxd8GDaKCEsIuwDs7lBOT61fqxS1fc1OgHs9tu9YaL1jefE0K2yeI3xdIums0NSY9Zd476uS2Nqt00DE3Lrw5TZH3rc7vf8faLfWKuv03O-UZYiXw7zwWbFl26MM3R-HasFKdP8W2d41Z81cWu-rlkFFcv1fpB3365fb5zYFUeuKjeJkSwiDqF3yqofO__gA/http%3A%2F%2Fpnda.io%2Fguide.

This is the illustration of my environment, I selected PNDA pico flavor cause my environment is small scale and experimental.

<blob.png>

The platform of my environment is OpenStack(Q version), so I build the environment step by step under "Creating PNDA on OpenStack".

All components' version are up-to-date, downloaded from github.com, including pndaproject/pnda, pndaproject/platform-salt and pndaproject/pnda-cli. 

I use pnda-cli.py to launch PNDA cluster deployment with all preparations ready. Then all PNDA cluster's VMs are running and the scripts of PNDA are executing. But there is an error when executing script for "sudo CLUSTER=zte-pnda salt-run --log-level=debug state.orchestrate orchestrate.pnda". 

The error is:

ID: hdp-start_hbase_rest

 Function: cmd.run

 Name: service hbase-rest stop || echo already stopped; service hbase-rest start

Result: False

  Comment: Command "service hbase-rest stop || echo already stopped; service hbase-rest start" run

  Started: 06:54:57.416991

 Duration: 133.36 ms

  Changes:

That means "hbase-rest" service can not be launched, and I try to resolve this problem.

  

service hbase-rest start

Job for hbase-rest.service failed because the control process exited with error code. See "systemctl status hbase-rest.service" and "journalctl -xe" for details

The reason is that item "JAVA_HOME" in hbase-env.sh is not configured correctly on the node of zte-pnda-hadoop-mgr-1. Should it be configured by salt?

# The java implementation to use.  Java 1.7+ required.

# export JAVA_HOME=/usr/java/jdk1.6.0/

So I set the right path of JAVA, then "hbase-rest" service is running.

[root@zte-pnda-hadoop-mgr-1 srv]# systemctl status  hbase-rest

● hbase-rest.service - HBase rest service

   Loaded: loaded (/usr/lib/systemd/system/hbase-rest.service; static; vendor preset: disabled)

   Active: active (running) since Fri 2019-04-26 07:20:55 UTC; 13s ago

  Process: 15599 ExecStart=/usr/hdp/current/hbase-master/bin/hbase-daemon.sh start rest -p 20550 --infoport 20551 (code=exited, status=0/SUCCESS)

 Main PID: 15615 (bash)

   CGroup: /system.slice/hbase-rest.service

           ├─15615 bash /usr/hdp/current/hbase-master/bin/hbase-daemon.sh --config /usr/hdp/current/hbase-master/bin/../conf foreground_start rest -p 20550 --infoport...

           └─15629 /usr/share/java/jdk1.8.0_131/bin/java -Dproc_rest -XX:OnOutOfMemoryError=kill -9 %p -Dhdp.version=2.6.4.0-91 -XX:+UseConcMarkSweepGC -Dhbase.log.di...

Apr 26 07:20:54 zte-pnda-hadoop-mgr-1 systemd[1]: hbase-rest.service holdoff time over, scheduling restart.

Apr 26 07:20:54 zte-pnda-hadoop-mgr-1 systemd[1]: Starting HBase rest service...

Apr 26 07:20:54 zte-pnda-hadoop-mgr-1 hbase-daemon.sh[15599]: starting rest, logging to /usr/hdp/current/hbase-master/bin/../logs/hbase-root-rest-zte-pnda-had...gr-1.out

Apr 26 07:20:55 zte-pnda-hadoop-mgr-1 systemd[1]: Started HBase rest service.

Hint: Some lines were ellipsized, use -l to show in full.

But when I execute the script "sudo CLUSTER=zte-pnda salt-run --log-level=debug state.orchestrate orchestrate.pnda" again, there is still an error occured:

    ID: orchestrate-pnda-create_master_dataset

    Function: salt.state

      Result: False

     Comment: Run failed on minions: zte-pnda-hadoop-edge

              Failures:

                  zte-pnda-hadoop-edge:

                  ----------

                            ID: master-dataset-create_hdfs_pnda_home

                      Function: cmd.run

                          Name: hdfs dfs -mkdir /user/pnda && hdfs dfs -chown pnda:pnda /user/pnda && hdfs dfs -chmod 770 /user/pnda

                        Result: False

                       Comment: Command "hdfs dfs -mkdir /user/pnda && hdfs dfs -chown pnda:pnda /user/pnda && hdfs dfs -chmod 770 /user/pnda" run

                       Started: 07:23:29.456002

                      Duration: 5483.784 ms

                       Changes:

I appreciate if any of you can tell me what's wrong with it and the solution.

By the way, when I try to open PNDA console on http://secure-web.cisco.com/1terjfe5tqnNK9zU0w99dHlMBYifbmg9lM5ikGbi5YP2zceRHNNPbm8rFXMjHxpwnuvS4v0SIoZGB-eut9iOqdQtL1Sm_SrGWD06HT0eJLRdrWuQWocDwf2OeqKtBoQ_VoccCt3z6j11bz_GZtH1BcQUAbKZnkWWSxTVn7psJpaExnwMWaGPse5OHYmC5U9wO6EGc17l8rOflHovVjWmtkJztX_ij2wwe_CosYI1lREEhfOwLGySyNqxG_z4Oxvdv1cU9_UEBUikABnL2AXCd5DEKbK-g3CXUOhYNXqjMXyBrED2xHkAGC2k6KOSBamlV/http%3A%2F%2Fzte-pnda-hadoop-edge-Node, user name and password are required. I try some combinations, such as pnda/pnda, dev1/dev1, prod1/prod1, but none of them is right. Could you tell me the reason.

<blob.png>

Looking forward to your reply soon! Thanks a lot!

Robin.