Apache Zeppelin Documentation

(A part of my Apache Zeppelin “project”.)

This is my conspectus of https://zeppelin.apache.org/docs/latest/.

• What is dynamic form?
  • Apache Zeppelin dynamically creates input forms.
  • Depending on language backend, there're two different ways to create dynamic form. Custom language backend can select which type of form creation it wants to use.
  • Forms can have different scope (paragraph or note).
    • Paragraph scope - available in current paragraph (=where the form is declared?)
    • Note scope - available in all paragraphs regardless of which paragraph has code to create these forms.

…

…

…

%table

…

The %network directive ⇒ Zeppelin treats the output as a graph.

…

%angular directive

println("%angular <h3>Hello world</h3>")

// bind my 'object' as angular scope variable 'name' in current notebook.
z.angularBind(String name, Object object)
 
// bind my 'object' as angular scope variable 'name' in all notebooks related to current interpreter.
z.angularBindGlobal(String name, Object object)
 
// unbind angular scope variable 'name' in current notebook.
z.angularUnbind(String name)
 
// unbind angular scope variable 'name' in all notebooks related to current interpreter.
z.angularUnbindGlobal(String name)

…

// register for angular scope variable 'name' (notebook)
z.angularWatch(String name, (before, after) => { ... })
 
// unregister watcher for angular variable 'name' (notebook)
z.angularUnwatch(String name)
 
// register for angular scope variable 'name' (global)
z.angularWatchGlobal(String name, (before, after) => { ... })
 
// unregister watcher for angular variable 'name' (global)
z.angularUnwatchGlobal(String name)

…

…

…

…

…

…

• Also a difference in term of scope:
  • Front-end API limits the Angular object binding to a paragraph scope
  • Back-end API allows you to bind an Angular object at the global or note scope.

https://zeppelin.apache.org/docs/0.11.0/usage/interpreter/overview.html

The concept of Zeppelin interpreter allows any language or data-processing backend to be plugged into Zeppelin.

Supported interpreters:

• Scala with Apache Spark (default)
• Python with Apache Spark
• SQL
• Hive
• JDBC
• Markdown
• Shell
• …

A Zeppelin interpreter is a plug-in which enables Zeppelin users to use a specific language/data-processing backend. For example, to use Scala code in Zeppelin, you would use the %spark interpreter.

You can create multiple interpreters for the same engine with different interpreter setting. e.g. You can create spark2 for Spark 2.x and create spark1 for Spark 1.x.

For each paragraph you write in Zeppelin, you need to specify its interpreter first via %interpreter_group.interpreter_name. e.g. %spark.pyspark, %spark.r.

• If you specify interpreter, you can also pass local properties to it (if it needs them).
  • This is done by providing a set of key/value pairs, separated by comma, inside the round brackets right after the interpreter name.
  • If key or value contain characters like =, or ,, then you can either escape them with \ character, or wrap the whole value inside the double quotes
  • Example:

    • %cassandra(outputFormat=cql, dateFormat="E, d MMM yy", timeFormat=E\, d MMM yy)

Every interpreter belongs to an Interpreter Group. Interpreter Groups are units of interpreters that run in one single JVM process and can be started/stopped together.

In the Interpreter Settings, one can choose one of the following interpreter binding modes:

• shared
  • every note/user using this interpreter will share a single interpreter instance
• scoped
  • can be used under 2 dimensions:
    • per user
    • per note
• isolated
  • can be used under 2 dimensions:
    • per user
    • per note

More info: https://zeppelin.apache.org/docs/0.11.0/usage/interpreter/interpreter_binding_mode.html

By default, Zeppelin will register and display all the interpreters under folder $ZEPPELIN_HOME/interpreters. But you can configure property zeppelin.interpreter.include to specify what interpreters you want to include or zeppelin.interpreter.exclude to specify what interpreters you want to exclude. Only one of them can be specified, you can not specify them together. (–> why?)

…

…

• shared
  • …
• scoped
  • …
• isolated
  • …

When your code requires external library, instead of doing download/copy/restart Zeppelin, you can easily do following jobs in this menu.

• Load libraries recursively from Maven repository
• Load libraries from local filesystem
• Add additional maven repository

This can be done from screen “Interpreters” (accessible from menu point “Interpreters” of the navigation bar on the main screen).

• Worked for me:
  • On my Linux:
    • setting the property “spark.jars” on the Screen “Interpreters”
  • On my Windows:
    • …

…

It is possible to register hooks - code which will be executed before or after paragraph code

• pre_exec
• post_exec

See: https://zeppelin.apache.org/docs/0.11.0/usage/interpreter/execution_hooks.html

https://zeppelin.apache.org/docs/0.11.0/usage/other_features/publishing_paragraphs.html

Paragraphs can be published in an external website (through iframe HTML tag) or simply opened in a new browser window. Simply copy the link (using menu point from the paragraph's menu) and use it. See:

…

…

• Compare Revisions

…

https://zeppelin.apache.org/docs/0.11.0/usage/other_features/cron_scheduler.html

• Crone scheduler can be used.
  • Crone configuration can be done from the Zeppelin UI

…

The zeppelin-context is a system-wide container for common utility functions and user-specific data.

…

• in many interpreters Zeppelin context is available through the variable z (e.g. in Spark)
• it can be used
  • for data exchange
    • e.g. String objects exchange between paragraphs:

%spark
z.put("maxAge", 83)

%jdbc(interpolate=true)

select * from bank where age = {maxAge}

For manipulation on interpreters and its settings. …

At the moment seems to be not very comprehensive. It is only these two calls available:

• get Zeppelin version
• change Zeppelin log level

This API allows manipulations (CRUD and start/stop/restart/etc) on :

• notes
• paragraphs
• permissions
• cron jobs
• version control

…

To manipulate config key/value pairs.

…

To manipulate credentials configuration.

…

Helium is a plugin system that can extend Zeppelin a lot. For example, you can write custom display system or install already published one in Helium Online Registry.

…