Skip to content. | Skip to navigation

Navigation

Focus on your applications!

Personal tools

You are here: Home / Blog

Blog

by Veit Schiele last modified Nov 12, 2019 02:29 PM

You can also subscribe to our blog as RSS 1.0, RSS 2.0 or Atom feed.

Data protection in times of Covid-19

by Veit Schiele last modified Apr 26, 2020 08:09 PM

Companies and organizations have data that they do not want to make available to others. They also have a special responsibility for their customers, partners and employees. Not being sovereign of this data means not only a loss of trust, but usually also commercial losses.

Show your customers, partners and employees that data protection is important to you and that you take responsibility to protect their privacy. Show that you have implemented the rules of the European General Data Protection Regulation (GDPR) from May 2018.

Therefore, do without Google services and use alternatives. Google makes money from the data you provide Google:

With your permission you give us more information about you, about your friends, and we can improve the quality of our searches. We don’t need you to type at all. We know where you are. We know where you’ve been. We can more or less know what you’re thinking about. [1]

This statement by the Google CEO, Eric Schmidt, is more relevant than ever. It can get scary when you think that a company knows more or less what you think about. The group only reveals part of this information if you still have a Google account – saved graphs and other evaluations will remain hidden from you.

In the following we would like to introduce you to some privacy-friendly alternatives to Google services:

… for your office work

  • Jitsi instead of Google Hangout, Zoom or Microsoft Teams
  • Mattermost instead of Slack
  • Nextcloud and OnlyOffice instead of Google Docs, Google Sheets, Google Slides, Google Calendar and Google Drive

… for your website

… for your apps

For further reading

Telearbeit und Mobiles Arbeiten
Information from the Federal Commissioner for Data Protection and Freedom of information (BfDI), January 2019
Top Tips for Cybersecurity when Working Remotely
Article by the European Union Agency for Cybersecurity (ENISA), March 2020
Home-Office? – Aber sicher!
Information from the Federal Office for Information Security (BSI), March 2020

[1]Google’s CEO: ‹The Laws Are Written by Lobbyists›, 2010.

How can I use Jitsi in compliance with data protection regulations?

by Veit Schiele last modified Apr 26, 2020 11:25 PM

Jitsi Meet is open source software that enables video conferencing with one or more participants. No registration is required for use, just a WebRTC capable web browser. The apps for Android and iOS are not recommend since they embed three trackers:

  • Google CrashLytics
  • Google Firebase Analytics
  • Amplitude

Note

There is an F-Droid version for Android without a tracker.

Note

The official Jitsi Meet Server (meet.jit.si) unfortunately uses Google Analytics and is therefore not recommended.

In addition to video conferencing with one or more participants, Jitsi also allows desktop or screen sharing, i.e. sharing the desktop or a certain window in order to show content to other participants. A chat function is also integrated to share text-based content with other participants.

Requirements

If you use Jitsi with your browser, you need a webcam that is at least integrated in most laptops today. The browser itself must support WebRTC.

Note

Chrome or Chromium is recommended by the Jitsi developers, since the least problems should occur there.

No end to end encryption

The data is encrypted from your browser via DTLS (Datagram Transport Layer Security) and SRTP (Secure Real-time Transport Protocol). However, WebRTC does not offer the possibility to encrypt video chats with more than two participants end to end, i.e. all data traffic is then decrypted on the Jitsi Meet server and is then visible to administrators. For the Jitsi servers hosted by us, our customers therefore receive an order data processing.

Tips

  • If the release of the local IP address for WebRTC is deactivated, e.g. by add-ons like uBlock Origin, you have to release it again.
  • If your outgoing network connections are filtered with a firewall, the ports TCP-443 (HTTPS) and UDP-10000 (video stream) should be released.
  • If Blokada is active, Jitsi Meet only works if you redirect Jits Meet in Settings → Apps → Installed apps. The device must then be restarted for the settings to take effect.
  • As an alternative to your web browser, you can also install Jitsi Meet Electron.
  • If you are an organiser, you should send the invitation so that audio and video are initially muted, e.g. https://meet.cusy.io/MyMeetup#config.startWithVideoMuted=true&config.startWithAudioMuted=true

Server setup

We operate the Jitsi Meet server in the following configuration:

  • 2 vCPUs, 5 GiB RAM

  • We have deactivated peer-to-peer mode because switching to server mode with 3 or more participants caused several problems. For this we changed the file jitsi-meet/config.js:

    p2p = {
        enabled = false,
    …
    }
    
  • In order to reduce the traffic and not to overload the clients, we took the following changes:

    • Limiting video resolution:

      constraints: {
          Video: {
              height: {
                  ideal: 480,
                  max: 480,
                  min: 240
              }
          }
      }
      
    • After the eighth participant, participation begins with a muted video:

      startVideoMuted: 8,
      
    • Only the last ten speakers are streamed:

      channelLastN: 10,
      
    • The blurring of the background is deactivated because it can lead to greater load on the users’ devices. For this we configured in jitsi-meet/interface_config.js:

      DISABLE_VIDEO_BACKGROUND = true;
      
  • In the standard configuration, the Jitsi video bridge (JVB) logs the IP addresses of the users in some log messages. We prevent this by setting the log level in ~/.jitsi-meet-cfg/jvb/logging.properties to WARNING.

Beuth University: Prototype for a medication app

by Veit Schiele last modified Dec 16, 2019 09:21 AM
Beuth University: Prototype for a medication app

Beuth University of Applied Sciences

For the Beuth University, we develop a prototype for a medication app.

The app is intended to improve the safety of the medication and in particular in the monitoring of ingestion rhythm and the knowledge of side effects and influences.

Not only the patients themselves should be able to use this app, but also relatives and caregivers.

In fact, there are already many apps that promise to meet the requirements. However, with more detailed research, they have significant shortcomings.

Professional quality

The professional quality of other apps is rarely discernible and, if the few reviews are taken as a basis, is usually very low. This is all the more problematic when apps promise to point out interactions and double prescriptions for medications with similar effects. For customers who rely on the fact that their app will warn them of dangers, for example with their self-medication requests, are likely to be at serious risk.

User groups

The apps also very rarely provide information about their user groups, neither about

  • Suitability for specific diseases/conditions
  • Suitability for gender, special age groups (or areas) etc.
  • Suitability for certain health professions and settings: clinical, outpatient, at home, …
  • Suitability for physiological and physical impairments, also not the support for TalkBack for Android and VoiceOver for iPhone.
  • Support for country-specific drugs and pack sizes

Privacy

The handling of user data is usually poor. The data protection declarations usually leave customers unclear as to what happens to their information. This is all the more problematic since over 80% of the apps transfer data to infrastructure providers such as Google, Facebook etc. Not even the encrypted transmission of user data was always guaranteed, especially not when data was transmitted by email. The few independent test procedures are unlikely to contribute to clarification, since they mostly rely on self-assessment.

elena international: Web-based planning tool for microgrids

by Veit Schiele last modified Dec 02, 2019 09:58 AM
For elena international we realise a web-based planning tool for microgrids where we use Jupyter notebooks and Voilà vuetify to develop presentation logic and user interactions fast, simple and robust.

elena international is a startup company that provides customised solutions for various stages of power system planning.

First we realise a walking skeleton of a web-based planning tool for microgrids.

The chosen software architecture allows elena the further development of its Julia libraries. In the Web tool the adoption of new features takes place in IPython notebooks with PyJulia. These notebooks can also be used to define the voila-vuetify widgets for interacting with users. Finally, with Voilà, these notebooks are converted into an interactive dashboard:

Voilà Dashboard

To harden the notebooks for production, on the one hand we write tests for each method, which run regularly with GitLab CI/CD, and on the other hand we activate logging for fault diagnosis and monitoring.

So we are not only answering the question Are Jupyter notebooks ready for production?, we also extend the possibilities of notebooks: they serve an editor system, in which not only texts can be written and media added, but also the presentation logic of scientific calculations and interactive widgets can be defined.

Are Jupyter notebooks ready for production?

by Veit Schiele last modified Nov 12, 2019 04:00 PM
Are Jupyter notebooks ready for production?

Jupyter Notebook

In recent years, there has been a rapid increase in the use of Jupyter notebooks, s.a. Octoverse: Growth of Jupyter notebooks, 2016-2019. This is a Mathematica- inspired application that combines text, visualisation, and code in one document. Jupyter notebooks are widely used by our customers for prototyping, research analysis and machine learning. However, we have also seen that the growing popularity has also helped Jupyter notebooks be used in other areas of data analysis, and additional tools have been used to run extensive calculations with them.

However, Jupyter notebooks tend to be inappropriate for creating scalable, maintainable, and long-lasting production code. Although notebooks can be meaningfully versioned with a few tricks, automated tests can also run, but in complex projects, mixing code, comments and tests becomes an obstacle: Jupyter notebooks can not be sufficiently modularized. Although notebooks can be imported as modules, these options are extremely limited: the notebooks must first be fully loaded into memory and a new module must be created before each cell can run in it.

As a result, it came to the first notebook war, which was essentially a conflict between data scientists and software engineers.

How To Bridge The Gap?

Notebooks are rapidly gaining popularity among data scientists and becoming the de facto standard for rapid prototyping and exploratory analysis. Above all, however, Netflix has created an extensive ecosystem of additional tools and services, such as Genie and Metacat. These tools simplify complexity and support a broader audience of analysts, scientists and especially computer scientists. In general, each of these roles depends on different tools and languages. Superficially, the workflows seem different, if not complementary. However, at a more abstract level, these workflows have several overlapping tasks:

data exploration occurs early in a project

This may include displaying sample data, statistical profiling, and data visualization

Data preparation

iterative task

may include cleanup, standardising, transforming, denormalising, and aggregating data

Data validation

recurring task

may include displaying sample data, performing statistical profiling and aggregated analysis queries, and visualising data

Product creation

occurs late in a project

This may include providing code for production, training models, and scheduling workflows

A JupyterHub can already do a good job here to make these tasks as simple and manageable as possible. It is scalable and significantly reduces the number of tools.

To understand why Jupyter notebooks are so compelling for us, we highlight their core functionalities:

  • A messaging protocol for checking and executing language-independent code
  • An editable file format for writing and capturing code, code output, and markdown notes
  • A web-based user interface for interactive writing and code execution and data visualisation

Use Cases

Of our many applications, notebooks are today most commonly used for data access, parameterization, and workflow planning.

Data access

First we introduced notebooks to support data science workflows. As acceptance grew, we saw an opportunity to leverage the versatility and architecture of Jupyter notebooks for general data access. Mid-2018, we started to expand our notebooks from a niche product to a universal data platform.

From the user’s point of view, notebooks provide a convenient interface for iteratively executing code, searching and visualizing data – all on a single development platform. Because of this combination of versatility, performance, and ease of use, we have seen rapid adoption across many user groups of the platform.

Parameterization

Along with increasing acceptance, we have introduced additional features for other use cases. From this work notebooks became simply paramatable. This provided our users with a simple mechanism to define notebooks as reusable templates.

Workflow planning

As a further area of notebook ​​applications, we have discovered the planning of workflows. They have the following advantages, among others:

  • On the one hand, notebooks allow interactive work and rapid prototyping and on the other hand they can be put into production almost without any problems. For this the notebooks are modularized and marked as trustworthy.
  • Another advantage of notebooks are the different kernels, so that users can choose the right execution environment.
  • In addition, errors in notebooks are easier to understand because they are assigned to specific cells and the outputs can be stored.

Logging

In order to be able to use notebooks not only for rapid prototyping but also for long-term productivity, certain process events must be logged so that, for example, errors can be diagnosed more easily and the entire process can be monitored. IPython Notebboks can use the logging module of the standard Python library or loguru, see also Jupyter-Tutoiral: Logging.

Testing

There have been a number of approaches to automate the testing of notebooks, such as nbval, but with ipytest writing notebook tests became much easier, see also Jupyter Tutorial: ipytest.

Summary

Over the last few years, we have been promoting close collaboration between Software Engineers and data scientists to achieve scalable, maintainable and production-ready code. Together, we have found solutions that can provide production-ready models for machine learning projects as well.