B028462 - B028463 - BIG DATA ARCHITECTURES e mutuatli
Corso di Studio: B070 - INGEGNERIA INFORMATICA
Università degli Studi di Firenze.
In this page you can find the new versions of the slides and video, while on moodle only some older version is accessible
Contenuti:
--Overview del corso e del Lab--Big data architectures
--Cloud: virtualizzazione, HA, DRS, FT, architetture
--Container: Docker, Kubernetes
--Big Data Analytics: prediction, early warning, anomaly detection
--IOT Architecture: AWS, Azure IOT, Google IOT, Snap4City IOT/IOE
--Tools: snap4city.org
--Big data computing, Performance analysis
--Big Data Analytics: prediction, early warning, anomaly detection, time series, deep learning, reinforced learning, XAI
--Visual Analytics: Business intelligence, Dashboard
--MLOps, machine learning operation
--Tools: Twitter vigilance (opzionale)
--Hadoop, MapReduce, Spark (opzionale)
--Batch and Data Stream processing: NIFI, Kafka, ETL, … (opzionale)
--Indexing and Search: SOLR, SOLR Sharded, Elastic Search (opzionale)
--Big Data Store: Hbase, Mongo (opzionale)
----- PROGRAMMA ---------
IL FATTO CHE ALCUNI VIDEO SONO MARCATI CON ANNI PRECEDENTI AL CORRENTE NON SIGNIFICA CHE SIANO OBSOLETI, MA SOLO CHE SONO STATI REGISTRATI IN TALE PERIODO, ED ESSENDO IL PROGRAMMA NON CAMBIATO, SONO ANCORA VALIDI.
OVERVIEW
- 2024: https://www.snap4city.org/download/video/disit/1-BigDataArc-2024-overvi…
- 2023: https://www.disit.org/files/bd/1-BigDataArc-2023-overview-v1-0.pdf
Cloud and container:
- 2024: virtualization and cloud: https://www.snap4city.org/download/video/disit/2-BigDataArc-2024-Parte-…
- 2024: containers, docker and kubernetes:
- 2023: https://www.disit.org/files/bd/2-BigDataArc-2023-Parte-1-Virtualization-n-Cloud-v4-5.pdf
- 2023: https://www.disit.org/files/bd/Lezione-Container-2023b.pdf
Architecture
- 2024: https://www.snap4city.org/download/video/disit/3-4new-BigDataArc-2024-D…
- 2023/4: https://www.disit.org/files/bd/4-bigdata-2023-IntroIOT-Arc-App-Dash-v1-8-annot3.pdf
- 2024: https://www.disit.org/files/bd/Node-RED-Snap4City-IoTApp.pdf
- 2023: https://www.disit.org/files/bd/3x-BigDataArc-2023-cloud-DataLake-vs-DataWarehouse-v0-6-annot.pdf
- Optional: potete anche accedere ai video del 2022 corso dell'Università di Firenze in ITALIANO, parte del corso di Sistemi Distribuiti del prof. P. Nesi:
- https://www.snap4city.org/download/video/20220428 1219-1.mp4
- https://www.snap4city.org/download/video/20220504 1438-1.mp4
- https://www.snap4city.org/download/video/20220505 1226-1.mp4
- https://www.snap4city.org/download/video/20220518 1436-1.mp4
- https://www.snap4city.org/download/video/20220525 1435-1.mp4
- https://www.snap4city.org/download/video/20220526 1222-2.mp4
- https://www.snap4city.org/download/video/20220601 1440-2.mp4
- https://www.snap4city.org/download/video/20220608 1445-1.mp4
Data Analytics, AI, XAI, overview, MLOps, machine Learning Operation
- 2024: https://www.snap4city.org/download/video/disit/3-4new-BigDataArc-2024-D…
- 2024: Machine Learning Operation, MLOps: https://www.snap4city.org/download/video/disit/MLOperation-ClearML-Snap…
- .
- 2023: https://www.disit.org/files/bd/Snap4City-2023-p4-Data-Analytics-Artific…
AI, TimeSeries and predictions, clustering, XAI
- 2024: time series: https://www.snap4city.org/download/video/disit/BigDataArc-2024-Time-Ser…
- 2024: clustering, XAI: https://www.snap4city.org/download/video/disit/BigDataArc-2024-clusteri…
- 2024: clustering, XAI: https://www.snap4city.org/download/video/disit/BigDataArc-2024-clusteri…
- 2024: clustering, XAI: https://www.snap4city.org/download/video/disit/BigDataArc-2024-clusteri…
- 2024: SUMO simulator, match demand vs offer: https://www.snap4city.org/download/video/disit/SUMO-match-demand-offer-…
- 2024: Reinforced Learning: https://www.snap4city.org/download/video/disit/RL-for-TLP-optimisation…
- 2023: https://www.disit.org/files/bd/Time-Series-Slides_v2.pdf
- 2023: https://www.disit.org/files/bd/Time-series-data-analysis.pdf
- 2023: https://www.disit.org/files/bd/Big-Data-Arc-Clustering-XAI-2023-v1-0.pdf
- 2023: https://www.disit.org/files/bd/Clustering.zip
Mobility and Transport, AI Data Analytics
- 2024: https://www.snap4city.org/download/video/disit/Data4Mobility-2024-v2.pdf
- 2024: https://www.snap4city.org/download/video/disit/MaaS-2024-v1.pdf
- 2024: https://www.snap4city.org/download/video/disit/TrafficFlowReconstructio…
- 2023:
Mobility Data Analytics Tensor Flow (Optional)
3D Digital Twin, Deck.GL
- 2023: https://www.snap4city.org/download/video/disit/SmartCityDigitalTwin-202…
- 2023: https://www.disit.org/files/bd/SmartCityDigitalTwin-2023-v1.pdf
- 2023: https://www.disit.org/files/bd/Big-Data-arc-2023-Deck-v0-1.pdf
- 2023: https://www.disit.org/files/bd/Big-Data-arc-2023-Deck-v0-1.mp4
- 2023: https://www.disit.org/files/bd/Big-Data-arc-2023-Deck-v0-1-example.zip
Object tracking Vision
NOSQL databases: Mongo, Hadoop, SOLR, ElasticSearch (Optional)
- https://www.disit.org/files/bd/NoSQL-Mongo-2023.pdf
- https://www.disit.org/files/bd/lex13-2020-10-20-hadoop.mp4
- https://www.disit.org/files/bd/lex14-2020-10-21-hadoop.mp4
- https://www.disit.org/files/bd/lex13-14-2020-10-20-21-3-Big-Data-Comput…
- https://www.disit.org/files/bd/lex15a-2020-10-23-addendum.mp4
- https://www.disit.org/files/bd/lex15b-2020-10-23-perf.mp4
- https://www.disit.org/files/bd/lex16-2020-10-27-perf-solr-elasticsearch…
---------------------------------- old versions --------------------------------
CORSO 2020: Big Data Architectures, overview
Video 2020: Lex1 (all'inizio vi sono svariati minuti di prove da saltare),
- finale Slide con annotazioni
Video 2020: Lex13, Lex14, Lex13-14,

Video 2020: Lex15a, Lex15b, Lex16, Lex 15b-16
Corso BigDataArc: Big Data, NoSQL, MongoDB
Video 2020: Video1, Video2
Corso BigDataArc: IOT generico e Snap4City: IOT and IOE, Architetture, Dashboard, scaling, etc.
Video 2020: video1, video2, video3, video4, video5, video6
Corso BigData Arc: data analytic
Video 2020: video1, video2, video3, video4
Corso BigDataArc: Container Docker
----------------- versioni precedenti------------------
Corso BigDataArc: Visual Analytics for IOT Data Flow and traffic flow
PER IOT App esercitazioni vedi slide: https://www.snap4city.org/download/video/barc19/iot/
oppure per il PDF: https://www.snap4city.org/download/video/barc19/iot/iot.pdf
Corso BigDataArc: Big Data Analytics, examples
Corso BigDataArc: 2018-19 elaborati e tesi
si veda anche https://www.disit.org/5492
-------------------------------------------------
Durante il corso saranno effettuate esercitazioni su cluster hadoop, mapreduce, IOT application, containers, elastic search, twitter vigilance.
Questo stesso portale DISIT.ORG contiene molte slide e contenuti nel tema Big Data Architetture, Cloud, IOT, etc. fate delle ricerche.
Modalità di verifica dell'apprendimento: un elaborato, small project
per elaborati, stage e Tesi si veda sopra le slide ma anche https://www.disit.org/5492
Si veda
https://www.disit.org
Https://www.km4city.org
https://www.disit.org/tv
https://www.snap4city.org
Docente: Paolo Nesi, mailto:paolo.nesi@unifi.it