<html> <div style=“font-size:xx-large; font-weight: 900;”>05.06.2024 Mittwoch</div> </html>

Appointments

None

Health

- Exercises
- - Rücken
  - - Morgens
  - - Abends
- - Augen
  - - Morgens
  - - Abends
- - Zunge
  - - Morgens
  - - Abends
- - Pr
  - - Morgens
  - - Abends

Tasks for the day

- datachecker
- - check out zeppelin project
  - https://github.com/apache/zeppelin.git
  - to /v/IdeaProjects/zeppelin
  - ZZEPPELIN_INTP_JAVA_OPTS=“-agentlib:jdwp=transport=dt_socket,server=y,suspend=y,address=6006”
- - check out sparktest project
  - git@78.47.83.54:/srv/git/sparktest.git
  - to /v/IdeaProjects/sparktest
- - export single outlier correction for Freudenstadt project
  - + implement in AcronCsvTransporter possibility to use own output variable name
    - “value_korr” should be default
  - + export all Freudenstadt data
    - exported files lie under: /v/testdata/zeppelin/export.csv/wsbe_*
  - - check corrected data month per month (full year will not work because of small memory)
    - whether valueKorr == value_korr or not
    - - wsbe_rueb3
    - - wsbe_rueb4
    - - wsbe_rueb5
    - - wsbe_rueb6
      - 1 diff:
        
        my correction:
        
        2022-06-25 15:40:00;wsbe_rueb6;3,3
        
        manual correction:
        
        2022-06-25 15:40:00;wsbe_rueb6;0
    - - wsbe_rueb11
    - - wsbe_zwiesel
- - try to configure Spark interpreter memory for Freudenstadt project
  - -Xmx16g worked, but: “Output is truncated to 64048000 bytes. Learn more about ZEPPELIN_INTERPRETER_OUTPUT_LIMIT”
    - ⇒ change zeppelin.interpreter.output.limit value in zeppelin-site.xml
- - implement BlockOutliersCleaner
  - - implement marking point as “isPositiveOutlier” (ColumnName.IS_POSITIVE_OUTLIER)
    - + implement simple scenario (current point is between start and end of outlier)
    - - eliminate false positives
      - - 2023-05-01T16:08:05
        
        multiple sequentially slopes should NOT be recognized
      - - 2023-05-08T09:38:15
      - - 2023-05-17T13:55:16
      - - 2023-05-24T13:02:28
        
        - ask Benjamin what can be done
- - implement SinglePointOutliersCleaner
  - - implement slope detection on multiple points
- - refactor GetSlopeFromOneRowUDF
  - - move maxAllowedSlope member to MarkSlopeFromOneRowUDF class
- - make a Transformer for “Blockdistanz” (see “1.2 Blockdistanz / Einstau Sonde”)
  - - implement HadSlopeMarkerTransformer
    - - check in current implementation
    - - implement MarkAsBetweenNegativeSlopeStartAndPositiveSlopeEndUDF
      - - test
        
        - test with data of one year
        
        - tests with real data
    - - clean up in HadSlopeMarkerTransformer and in all UDFs used there
      - - simplify getting/marking slopes using only
        
        SLOPE_TO_PREVIOUS_COLUMN_NAME_DEFAULT
        
        ⇒ getSlopeToPrevious using GetSlopeFromOneRowUDF()
        
        SLOPE_TO_NEXT_COLUMN_NAME_DEFAULT
        
        ⇒ getSlopeToNext using GetSlopeFromOneRowUDF()
  - - implement SlopeBasedBlankingDistanceDataTransformer
    - - implement ReplaceValuesWithinEntlastungEventUDF properly
      - ??? use gradient (as first solution) ⇒ GradientBasedBlankingDistanceDataTransformer
        
        (⇒ it should be not necessary because it is already calculated in HadSlopeMarkerTransformer.BETWEEN_NEGATIVE_SLOPE_START_AND_POSITIVE_SLOPE_END_MARKER_COLUMN_NAME_DEFAULT)
        
        ??? check how gradient can be calculated
        
        proposal 1:
        
        if (start point is near of the maxValue)
        
        if(t2-t1 < X min)
        
        if(slope > 0 && slope > maxSlope)
        
        ⇒ mark as “probableBlockingDistance”
        
        proposal 2 (older?):
        
        if (start point is near of the maxValue)
        
        if(there was a slope with “speed” ⇐ maxSpeed)
  - - define scenarios
    - Scenario 1. (simple)
      - runter und nach einer Weile wieder hoch
        
        Beispiel: Bild 1, zweite Lücke
    - Scenario 2.
      - runter, dann hoch, aber nicht bis zum Anschlag, dann ein paar mal hoch und runter, aber jedes Mal nicht bis zum Anschlag und nur ziemlich später wieder hoch bis zum Anschlag
        
        Beispiel: Bild 1, erste Lücke
    - Scenario 3
      - runter, dann relativ stabil, dann aber noch stark nach unten
        
        Besipel: Bild 3
    - Scenario 4
      - ask Benjamin
        
        letztes Bild im Abschnitt 1.2, letzte Lücke
        
        es ist unklar wie die rechte Seite von der Lücke korrigiert wurde
  - - test all defined scenarios
- - optimize building
  - set jar to “target” of sparktest
- - create a new project for transformers
- - test “1.1. Einfache Ausreißer”
  - test data is in /home/vkalaschnikow/Desktop/buero/Entwicklung/Plausibilitätsprogramm/Beispiele Messfehler Datenlücken
- - implement a graph for our data
  - - find out which graph libs exist
  - - decide how to handle “big data problem”
    - - find out which solutions exist
    - - implement 2-3 options
- - “data gaps analysis” → SparkTester.getStatsAboutMultipleIntervals()
  - - test Apache Zeppelin
    - - build zeppelin from IntelliJ
    - - Writing visualizations:
      - https://zeppelin.apache.org/docs/0.11.0/development/helium/writing_visualization_basic.html
- - Apache Spark
  - - check statistics possibilities
  - - check reporting possibilities

- rdconverter
- - write a “User Guide” for rdconverter
- - make a small “Project Developer Guide”
- - Clean up
  - - nuertingen
    - - use “data/in” as input directory
    - - use “vars” directory for configs
  - - nagold
    - - use “data/in” as input directory (make use of recursion!)
    - - check what can be done additionally
- - extend documentation
  - - CsvPreprocessor

- Lieb-ib

- Maven
- - Introduction to the Build Lifecycle

- Anleitung für neue Mitarbeiter
- Acron kennenlernen
- - Acron-Anleitung.docx
- - Beschreibung Datentransfer.docx
- Projekte zur Einarbeitung
  - SES
  - Sindelfingen Q3 2023
  - Aalen Q3 2023
…
…
…

- DWA-M 151
- - Entry Part 5 into my wiki
- - Summarize Part 6

Task pool

- [AA] Learning Angular
- - Angular Docs Understanding Angular
  - - Components
    - - Component styles
    - - Sharing data between child and parent directives and components
- - Tutorial: Your first Angular app
- - Angular Concepts

- [AB] Decide what I want to do
- - Search for business idea
  - - Checklist plugin
    - - check open source Task management systems
      - - OpenProject
    - - check open source CRMs
      - - check EspoCRM

- Check apps
- - something for Fleetmanagement
  - misc links:
- - Tracker for Traccar
  - - can it be profitable?
  - - what if the app would be contain an optional “Tasks” feature?
    - configuration
      - enable
      - server address
      - user
      - etc
    - simple user creation
- - Professional GPS tracker
  - IoT parameters

- Open Source Geschäftsmodelle

- make a plan for Avril
- - development
- - sales and marketing

Table of Contents

Appointments

Health

Tasks for the day

Task pool

Goals for the week