Overview
Red PNDA is aimed to be a smaller, simpler subset of PNDA; it provides a set of components providing a PNDA-like environment for development, education and demonstration. It’s more lightweight and designed to run on a laptop, enabling users to get familiar with the core data-ingest mechanism of PNDA (Kakfa/AVRO), as well as the data-exploration tools Jupyter, OpenTSDB and Grafana. It includes Apache Spark and Hbase but doesn’t include the ‘heavy’ components such as the Hadoop infrastructure and distributed processing.
Scenarios
- Scenario 1: Start PNDA
- Scenario 2: Explore Jupyter
Components
• Console Frontend - https://github.com/pndaproject/platform-console-frontend
• Console Backend
• Platform Testing
• Platform Libraries
• Kafka 0.11.0.0
• Jupyter Notebook
• Apache Spark 1.6.1
• Apache Hbase 1.2.0
• OpenTSDB 2.2.0
• Grafana 4.3.1
• Kafka Manager 1.3.3.6
• Example Kafka Clients
• Jmxproxy 3.2.0
Features
Red PNDA | - This framework provisions a minimal set of the PNDA (pnda.io) components to enable developers writing apps targeted at the full PNDA stack, to experiment with the PNDA components in a smaller, lightweight environment. Data exploration and app prototyping is supported using Jupyter and Apache Spark.
- Complete Documentation is available at https://github.com/pndaproject/red-pnda/blob/develop/README.md
|
Apache Kafka | |
Jupyter Notebook Data Exploration | Some sample notebooks available in Red PNDA: - A network-related dataset (BGP updates from the Internet) and an accompanying tutorial Juypter notebook named ‘Introduction to Big Data Analytics.ipynb’.
- ‘red-pnda-anom-detect.ipynb’ notebook is an Python implementation of a simple mu-sigma-based detection algorithm to identify packet losses in XR telemetry data captured from a simple multiple host topology. Such packet losses are often indications of unintentional black-holing of traffic. A small accompanying dataset is also provided to help people get started.
|
| Time Series Data Exploration | - Send data to OpenTSDB (a time series db) and visualize it in Grafana. Can be done via Jupyter notebook. See ‘red-pnda-anom-detect.ipynb’ jupyter notebook as an example.
|
Resources
