How to create Warp10 + Zeppelin configuration

19 views
Skip to first unread message

Fabien S.

unread,
Jul 29, 2021, 10:53:50 AM7/29/21
to Warp 10 users
Hello Group
Here is a little post How to create Warp10 + Zeppelin configuration

For proper reading,   article should be available on :


Here a little How To to create a Warp10 instance with a Zeppelin server.
The following configuration is for Warp10-ci docker image.
It means there is a read and write token already inserted : 'readTokenCI' and 'writeTokenCI'.

Warp10 and Zeppelin servers use the docker-host local mount point to store configurations and databases.

We need :

* a docker-compose.yml configuration file to run both server
* Warp10 configuration files
* Zeppelin configuration files.
* the Zeppelin plugin for both Warp10 and Zeppelin

## Configuring the docker-compose

The `docker-compose.yml` file is the following:


```yml
version: '2'

services:
warp10-alldb:
container_name: warp10-alldb
image: warp10io/warp10:2.8.1-ci
environment:
- WARP10_HEAP=6g
- WARP10_HEAP_MAX=6g
volumes:
- ./warp10-rt:/data
zeppelin:
# WarpScript interpreter plugin should be configured at: warp10-alldb:9377
container_name: zeppelin
image: apache/zeppelin:0.8.2
environment:
- ZEPPELIN_ADDR=0.0.0.0
- ZEPPELIN_PORT=8080
- ZEPPELIN_LOG_DIR=/zepp-rt/logs
- ZEPPELIN_NOTEBOOK_DIR=/zepp-rt/notebook
- ZEPPELIN_CONF_DIR=/zepp-rt/conf
volumes:
- ./zepp-rt:/zepp-rt/
- ./warp10-rt/warp10/lib:/zeppelin/interpreter/WarpScript
```

## Get the Warp10 configuration files

You can get the configuration Warp10 files with :

```bash
DOCKERW_ID=$(docker run --rm -d warp10io/warp10:2.8.1-ci)
echo $DOCKERW_ID
# You can explore container with docker exec $DOCKERW_ID ls -l --color /
# Create mandatory files
mkdir -p ./warp10-rt/warp10/logs
touch ./warp10-rt/warp10/logs/warp10.log
touch ./warp10-rt/warp10/logs/warpscript.out
mkdir -p ./warp10-rt/warp10/leveldb
# Wait the time Warp10 configure himsef
sleep 20
# Copy the configuration of Warp10
docker cp -L $DOCKERW_ID:/opt/warp10/etc ./warp10-rt/warp10/
docker cp -L $DOCKERW_ID:/opt/warp10/lib ./warp10-rt/warp10/
docker cp -L $DOCKERW_ID:/opt/warp10/jars ./warp10-rt/warp10/
docker cp -L $DOCKERW_ID:/opt/warp10/macros ./warp10-rt/warp10/
docker cp -L $DOCKERW_ID:/opt/warp10/warpscripts ./warp10-rt/warp10/
# Copy the configuration of Sensision
mkdir warp10-rt/sensision/
docker cp -L $DOCKERW_ID:/opt/sensision/etc ./warp10-rt/sensision/
docker cp -L $DOCKERW_ID:/opt/sensision/scripts ./warp10-rt/sensision/
mkdir ./warp10-rt/sensision/{logs,metrics,queued,targets}
touch ./warp10-rt/sensision/logs/sensision.pid
touch ./warp10-rt/sensision/logs/sensision.log

# Stop and remove the running Warp10 docker container
docker stop $DOCKERW_ID
```

Edit the configuration file `./warp10-rt/warp10/etc/conf.d/80--plugins.conf` and add the following:

```properties
warp10.plugin.zeppelin=io.warp10.plugins.zeppelin.ZeppelinWarp10Plugin
```

Edit the configuration file `./warp10-rt/warp10/etc/conf.d/ ` and change the following:

```properties
egress.clients.expose = true
```




## Get the Zeppeling plugin for both Zeppeling and Warp10

```bash
mv warp10-plugin-zeppelin-1.0.4-uberjar.jar warp10-rt/warp10/lib/

```


## Get the Zeppelin configuration files


```bash
DOCKERZ_ID=$(docker run --rm -d -v $PWD/warp10-rt/warp10/lib:/zeppelin/interpreter/WarpScript apache/zeppelin:0.8.2)
echo $DOCKERZ_ID
# You can explore container with docker exec $DOCKERZ_ID ls -l --color /
# Copy the configuration of Zeppelin
sleep 4
mkdir -p ./zepp-rt/conf/
docker cp -L $DOCKERZ_ID:/zeppelin/conf ./zepp-rt
# Stop and remove the running Warp10 docker container
docker stop $DOCKERZ_ID
```


## Docker magics !

```bash
docker-compose up
```

You will find the IP of the servers with the command line: `docker network inspect test-w10_default`

You can use WarpStudio with the warp10-alldb IP on port 8081.

You can use Zeppelin with the zeppelin IP on port 8080

Enjoy.


## Configure Zeppelin server

Go on http://ZEPPELIN_DOCKER_IP:8080/#/interpreter and configure the interpreter WarpScript with:

* Connect to existing process
* Host: warp10-alldb
* Port: 9377


## In case of error

You can remove all the docker jobs with:

```bash
docker-compose down && docker container prune && docker network prune
```

## Enjoy Warp10 + Zeppelin

Now you can enjoy with the first script:

```
%WarpScript

ZLEVELS

'%09' URLDECODE 'char.tab' STORE
'%0A' URLDECODE 'char.newline' STORE

// Create a List of string for the result (a table) with header "ts and val"
[ '#table ts' $char.tab 'value' ] '' JOIN
[ SWAP ]

[ [ 1000 14.2 ] $char.tab JOIN ] APPEND
[ [ 2000 42.12 ] $char.tab JOIN ] APPEND
[ [ 3000 33.10 ] $char.tab JOIN ] APPEND
[ [ 6000 20.7 ] $char.tab JOIN ] APPEND

$char.newline JOIN

```
Reply all
Reply to author
Forward
0 new messages