vwiki

User Tools

Site Tools

Trace:
apache_zeppelin_documentation

Differences

This shows you the differences between two versions of the page.

Link to this comparison view

Both sides previous revisionPrevious revision
Next revision		Previous revision
apache_zeppelin_documentation [2024/03/12 11:26] – [Dependency Management for Interpreter] kalaschnikowapache_zeppelin_documentation [2024/03/12 16:04] (current) – [Apache Zeppelin Documentation] kalaschnikow
Line 1: Line 1:
 +====== Apache Zeppelin Documentation ======
 +
 +(A part of my [[Apache Zeppelin]] "project".)
 +
 +This is my conspectus of https://zeppelin.apache.org/docs/latest/. 
 +
 +
 +===== Dynamic forms =====
 +  * [[https://zeppelin.apache.org/docs/0.11.0/usage/dynamic_form/intro.html|What is dynamic form?]]
 +    * Apache Zeppelin dynamically creates input forms.
 +    * Depending on language backend, there're two different ways to create dynamic form. Custom language backend can select which type of form creation it wants to use. 
 +    * Forms can have different scope (paragraph or note). 
 +      * Paragraph scope - available in current paragraph (=where the form is declared?)
 +      * Note scope - available in all paragraphs regardless of which paragraph has code to create these forms.
 +
 +
 +===== Display System =====
 +
 +==== Text ====
 +
 +...
 +
 +
 +==== HTML ====
 +
 +...
 +
 +==== Markdown ====
 +
 +...
 +
 +==== Table ====
 +
 +''%table''
 +
 +...
 +
 +==== Network ====
 +
 +The ''%network'' directive => Zeppelin treats the output as a graph.
 +
 +...
 +
 +
 +==== Backend Angular API  ====
 +
 +''%angular'' directive
 +
 +=== Print AngularJS view ===
 +
 +<code>println("%angular <h3>Hello world</h3>")</code>
 +
 +=== Bind/unbind variables ===
 +<code scala>
 +// bind my 'object' as angular scope variable 'name' in current notebook.
 +z.angularBind(String name, Object object)
 +
 +// bind my 'object' as angular scope variable 'name' in all notebooks related to current interpreter.
 +z.angularBindGlobal(String name, Object object)
 +
 +// unbind angular scope variable 'name' in current notebook.
 +z.angularUnbind(String name)
 +
 +// unbind angular scope variable 'name' in all notebooks related to current interpreter.
 +z.angularUnbindGlobal(String name)
 +</code>
 +...
 +
 +=== Watch/unwatch variables ===
 +<code scala>
 +// register for angular scope variable 'name' (notebook)
 +z.angularWatch(String name, (before, after) => { ... })
 +
 +// unregister watcher for angular variable 'name' (notebook)
 +z.angularUnwatch(String name)
 +
 +// register for angular scope variable 'name' (global)
 +z.angularWatchGlobal(String name, (before, after) => { ... })
 +
 +// unregister watcher for angular variable 'name' (global)
 +z.angularUnwatchGlobal(String name)
 +</code>
 +...
 +
 +
 +==== Frontend Angular API  ====
 +
 +...
 +
 +=== Bind/Unbind variables ===
 +
 +...
 +
 +=== Run paragraph ===
 +
 +...
 +
 +
 +=== Overriding dynamic form with Angular Object ===
 +
 +...
 +
 +=== Feature matrix comparison (between back-end and front-end API) ===
 +
 +...
 +
 +  * Also a difference in term of scope: 
 +    * Front-end API limits the Angular object binding to a //**paragraph scope**// 
 +    * Back-end API allows you to bind an Angular object at the //**global or note scope**//.
 +
 +
 +===== Interpreter in Apache Zeppelin =====
 +
 +https://zeppelin.apache.org/docs/0.11.0/usage/interpreter/overview.html
 +
 +The concept of Zeppelin interpreter allows any language or data-processing backend to be plugged into Zeppelin.
 +
 +Supported interpreters:
 +  * Scala with Apache Spark (default)
 +  * Python with Apache Spark
 +  * SQL
 +  * Hive
 +  * JDBC
 +  * Markdown
 +  * Shell
 +  * ...
 +
 +==== What are Zeppelin interpreters? ====
 +
 +A Zeppelin interpreter is a plug-in which enables Zeppelin users to use a specific language/data-processing backend. For example, to use Scala code in Zeppelin, you would use the ''%spark'' interpreter.
 +
 +You can create multiple interpreters for the same engine with different interpreter setting. e.g. You can create spark2 for Spark 2.x and create spark1 for Spark 1.x.
 +
 +
 +For each paragraph you write in Zeppelin, you need to specify its interpreter first via ''%interpreter_group.interpreter_name''. e.g. ''%spark.pyspark'', ''%spark.r''.
 +
 +  * If you specify interpreter, you can also pass local properties to it (if it needs them). 
 +    * This is done by providing a set of key/value pairs, separated by comma, inside the round brackets right after the interpreter name. 
 +    * If key or value contain characters like =, or ,, then you can either escape them with \ character, or wrap the whole value inside the double quotes
 +    * Example:
 +      * <code>%cassandra(outputFormat=cql, dateFormat="E, d MMM yy", timeFormat=E\, d MMM yy)</code>
 +
 +==== What are interpreter groups? ====
 +
 +Every interpreter belongs to an Interpreter Group. Interpreter Groups are units of interpreters that run in one single JVM process and can be started/stopped together. 
 +
 +==== Interpreter binding modes ====
 +
 +In the Interpreter Settings, one can choose one of the following interpreter binding modes:
 +  * ''shared''
 +    * every note/user using this interpreter will share a single interpreter instance
 +  * ''scoped''
 +    * can be used under 2 dimensions: 
 +      * per user 
 +      * per note
 +  * ''isolated'' 
 +    * can be used under 2 dimensions: 
 +      * per user 
 +      * per note
 +
 +More info: https://zeppelin.apache.org/docs/0.11.0/usage/interpreter/interpreter_binding_mode.html
 +
 +==== Choose Interpreters ====
 +
 +By default, Zeppelin will register and display all the interpreters under folder ''$ZEPPELIN_HOME/interpreters''. But you can configure property ''zeppelin.interpreter.include'' to specify what interpreters you want to include or ''zeppelin.interpreter.exclude'' to specify what interpreters you want to exclude. Only one of them can be specified, you can not specify them together. (--> why?) 
 +
 +===== User Impersonation =====
 +
 +...
 +
 +===== Interpreter Binding Mode =====
 +
 +...
 +
 +  * shared
 +    * ...
 +  * scoped
 +    * ...
 +  * isolated
 +    * ...
 +
 +===== Dependency Management for Interpreter =====
 +
 +When your code requires external library, instead of doing download/copy/restart Zeppelin, you can easily do following jobs in this menu.
 +
 +  * Load libraries recursively from Maven repository
 +  * Load libraries from local filesystem
 +  * Add additional maven repository
 +
 +
 +This can be done from screen "Interpreters" (accessible from menu point "Interpreters" of the navigation bar on the main screen).
 +
 +==== P.S. ====
 +  * Worked for me: 
 +    * On my Linux: 
 +      * setting the property "spark.jars" on the Screen "Interpreters"
 +    * On my Windows: 
 +      * ...
 +
 +==== Installing interpreters ====
 +
 +...
 +
 +==== Interpreter Execution Hooks ====
 +
 +It is possible to register hooks - code which will be executed before or after paragraph code
 +  * ''pre_exec''
 +  * ''post_exec''
 +
 +See: https://zeppelin.apache.org/docs/0.11.0/usage/interpreter/execution_hooks.html
 +
 +
 +===== Other Features =====
 +
 +==== Publishing Paragraphs ====
 +https://zeppelin.apache.org/docs/0.11.0/usage/other_features/publishing_paragraphs.html
 +
 +Paragraphs can be published in an external website (through ''iframe'' HTML tag) or simply opened in a new browser window. Simply copy the link (using menu point from the paragraph's menu) and use it. See:
 +
 +==== Customizing Apache Zeppelin homepage ====
 +...
 +
 +==== Notebook actions ====
 +...
 +  * Compare Revisions
 +...
 +
 +
 +==== Running a Notebook on a Given Schedule Automatically ====
 +https://zeppelin.apache.org/docs/0.11.0/usage/other_features/cron_scheduler.html
 +
 +  * Crone scheduler can be used.
 +    * Crone configuration can be done from the Zeppelin UI
 +
 +...
 +
 +==== Zeppelin-Context ====
 +
 +The zeppelin-context is a system-wide container for common utility functions and user-specific data.
 +
 +...
 +
 +  * in many interpreters Zeppelin context is available through the variable ''z'' (e.g. in Spark)
 +  * it can be used
 +    * for data exchange 
 +      * e.g. String objects exchange between paragraphs:
 +<code>
 +%spark
 +z.put("maxAge", 83)
 +</code>
 +
 +<code>
 +%jdbc(interpolate=true)
 +
 +select * from bank where age = {maxAge}
 +</code>
 +
 +
 +===== REST API =====
 +
 +==== Interpreter API ====
 +
 +For manipulation on interpreters and its settings.
 +...
 +
 +
 +==== Zeppelin Server REST API ====
 +
 +At the moment seems to be not very comprehensive. It is only these two calls available:
 +
 +  * get Zeppelin version
 +  * change Zeppelin log level
 +
 +==== Zeppelin Notebook REST API ====
 +
 +This API allows manipulations (CRUD and start/stop/restart/etc) on :
 +  * notes
 +  * paragraphs
 +  * permissions
 +  * cron jobs
 +  * version control
 +
 +
 +==== Zeppelin Notebook Repository API ====
 +
 +...
 +
 +==== Zeppelin Configuration REST API ====
 +
 +To manipulate config key/value pairs.
 +
 +...
 +
 +
 +==== Zeppelin Credential REST API ====
 +
 +To manipulate credentials configuration.
 +
 +...
 +
 +
 +
 +==== Zeppelin Helium REST API ====
 +
 +Helium is a plugin system that can extend Zeppelin a lot. For example, you can write custom display system or install already published one in Helium Online Registry. 
 +
 +...
 +
 +
 +