Datami

Edit, visualise, share 
 and contribute to datasets 
 the easy way

Julien Paris
fullstack developer
co-founder of tech cooperative multi

March 20th 2023
Plan by pressing esc
Versions : fr / en

Help your publics with your data

What are the challenges ?

Challenge #1

Make your data intelligible

 
 

Your raw data
without Datami

Your data enhanced
with Datami

Challenge #2

Make updating and contributing easier

Handling data is a matter of habits, facilitating their manipulation for the greatest number means that you have to adapt to the habits of the greatest number

To allow everyone to easily contribute to the data, the view in the form of a table is still the most commonly adopted today.

 

Challenge #3

Make all of the above with a tight budget

When you are a small or medium-sized structure (association, community, etc.) and you produce data of general interest, it can be complicated to highlight them on your site or to call on your community to put them up to date

Lack of resources, lack of skills, lack of time…

The cost of the usual technical solutions for sharing / viewing / contributing to data is often explained by the technical complexity of these functionalities

Another cost is related to the need to set up dedicated servers in backend or very specific configurations

The maintenance of applications or servers often generates significant costs

Datami’s original architecture makes it possible to get rid of a large part of these backend server costs while allowing customization dataset by dataset

Enhance your open datasets with Datami

 

Your spreadsheets transformed into maps

Datami allows you to create customized interactive maps

Your territorial data can be visualized in geographical form, whatever their themes

 

View your data from all angles

Your data can be viewed as spreadsheets, maps, miniature or detailed lists of records, or graphs

All views are interactive and customizable to highlight all the specifics of your data

 

Adapt Datami
to your needs

 
 

Your sketches
and your datasets

… implemented and enhanced
with Datami

You control your data
Datami makes it understandable

Datami’s architecture is designed as an interface between the citizen and your database in order to facilitate the link between citizens and open data producers

Datami doesn’t store your data : your data stays within the tool of your choice (Github, Gitlab, database or API).
Using Datami is more affordable because you don’t have to install nor maintain a dedicated backend.

Integrate Datami into your
site and your partners

Datami uses widgets technology: turnkey and customizable modules that you can add to an existing site

Datami widgets are open source, simple to copy and paste, without subscription, without additional cost

 

A simplified process
for contributory data updates

Empower your teams and audiences
to improve data

Datami includes a contribution and moderation system

Based on the Git language, Datami widgets allow you to keep control of your data and manage contributions, without creating an account to suggest improvements

 

Structure your data
to make it interoperable

Your data may be associated with files to structure it, such as data schema files

By associating your data set with a data schema respecting international standards you ensure that they can be correctly reused and improved

 

A management tool

Drive your field actions through data

Exploring your data allows you to better understand and manage your actions in the field

Datami allows you to set up personalized interactive data-visualizations, in order to make your data easily explorable

 

An open source tool

A free and multi-purpose software

 

Discover the interfaces

Datami is fully auditable and reusable free licensed software

To discover and learn how to use Datami you can:

The official website

On our official website you will find videos of presentation, examples, as well as our blog space

The site is translated into French and English

The source code

Our source code is on Gitlab under open source license

Do not hesitate to report bugs to us by suggesting issues, or to give ideas for new features on our roadmap

A mirror repo is also automatically synced to Github

Don’t forget to leave a little ⭐️ if you like the project!

 

The technical documentation site

Also visit our dedicated documentation site

The site is translated into French and English

You will find different sections there: technical principles, tutorials, examples, description of the different widgets and their configuration elements…

The stack

References

On Le Comptoir du Libre

Visit this page to leave a comment on Datami!

 

On AlternativeTo

Visit this page to leave a comment on Datami!

 

Service offer

The Datami offer by the multi cooperative

In order to make Datami as accessible as possible, our principle is to share design and development costs

All developments - even minimal ones - contributing to improving Datami thus ultimately benefit all users.

 

Setting up Datami

 

We offer an economic package of a few days only for support in setting up Datami

We can help you configure Datami and advise you in your data recovery project

Free software

 

Datami is a 100% open source tool

You can use Datami as it is based on the documentation, you are free!

Custom Developments

 

Do not hesitate to contact us to tell us about your needs to establish a personalized quote

We are committed to continuing to develop and improve Datami, and we also offer services in data science and data engineering

Installation guide

The source code

Clone the source code from Datami’s Gitlab repo

git clone git@gitlab.com:multi-coop/datami-project/datami.git

Then navigate to the folder

cd datami

Installation

Install npm 8.3.2

npm install -g npm@8.3.2

… or use nvm

brew install nvm # sur mac
nvm use

Install the dependancies

npm install

Create a local .env file based on example.env

cp example.env .env

Run Datami

Run the script

npm run serve

In your browser

http://localhost:8080

Liste d’exemples de widgets

Run the local server

nvm use
npm run http

In your browser

http://localhost:8180/html-tests/

Benchmark

There are several data visualization and editing solutions
sharing similarities with Datami

Here are some of the most popular solutions

Those benchmarks are given for information only

Do not hesitate to let us know by writing to
contact@multi.coop
if you wish to make any corrections or additions

Datavisualisation tools

Solution Solution type Langages Difficulty Saas Official website
Gogocarto Open source Custom Easy Yes https://gogocarto.fr/projects
Umap Open source Custom Facile Yes https://umap.openstreetmap.fr/
Lizmap Open source Propre langage de requête Medium Yes https://www.lizmap.com
Apache Superset Open source SQL Medium Yes https://superset.apache.org/
Apache Zeppelin Open source Several programmation languages Hard No https://zeppelin.apache.org/
BIRT Open source Java, JavaScript Hard No https://www.eclipse.org/birt/
FineReport Open source Java, JavaScript Medium No https://www.finereport.com/en
Grafana Open source Custom request language Hard No https://grafana.com/
Metabase Open source SQL Easy Yes https://www.metabase.com/
Redash Open source SQL Medium No https://redash.io/
Datasette Open source SQL Medium No https://datasette.io/
LightDash Open source Dbt Medium Yes https://www.lightdash.com/
Google Data Studio Free (but not open) SQL Easy Yes https://datastudio.google.com/
Datawrapper Commercial API, CSV, GSheet Easy Yes https://www.datawrapper.de/
Google Looker Commercial LookML Hard Yes https://looker.com/
Microsoft Power BI Commercial DAX et M Medium Yes https://powerbi.microsoft.com/
QlikView Commercial Custom script language Hard Yes https://www.qlik.com/
Tableau Commercial Custom Medium Yes https://www.tableau.com/

Online editing tools

Solution Solution type Langages Difficulty Saas Targeted public Official webiste
Apache Zeppelin Open source Scala, Python, R, SQL Hard No Developers and professional users https://zeppelin.apache.org/
Baserow Open source Python, Javascript, Vue.js Moyen Yes Developers and professional users https://baserow.io/
Grist Open source Python Easy Yes Businesses, non-profit organizations, governments, universities, researchers https://getgrist.com/
Metabase Open source Java, Clojure Medium Yes Startups, businessees, non-profit organizations https://www.metabase.com/
LockoKit Open source Medium No Developers and professional user https://locokit.io/
NoCodB Open source Javascript, Node.js Easy Yes Developers and professional users https://nocodb.com/
Gsheet - None Easy Yes Companies, teams, freelancers, SMEs https://www.google.com/sheets/about/
Airtable Commercial None Easy Yes Companies, teams, freelancers, SMEs https://airtable.com/
Qlikview Commercial None Medium Yes Large corporations, financial institutions https://www.qlik.com/us/

Benchmark features

Among all the solutions we have just listed, some can be compared with Datami’s main features.

Solution Open source Readyness Table view Cards view Map view Dataviz view Edition Moderation Configuration interface Data sources Backend Widget Official website
Datami ⭐⭐ ❌ (for now) API ext. (Git) Git platforms / APIs Website
Metabase ⭐⭐ ⚠️ SQL, connectors server / APIs Website
Gogocarto ⭐⭐⭐ proper server / APIs Website
Lizmap ⚠️ proper server / ❓ Website
Umap ⭐⭐⭐ server / ❓ Website
Grist SQL server / APIs Website
Baserow ⭐⭐ SQL server / ❓ Website
LockoKit ⭐⭐ proper server / ❓ Website
NoCodB SQL server / ❓ Website
Apache Superset ⭐⭐ SQL server / Saas Website
Datawrapper 🔒 ⭐⭐⭐ SQL, connectors Saas Website
Airtable 🔒 ⭐⭐⭐ ⚠️ ⚠️ proper, connectors Saas Website
Gsheet 🔒 ⭐⭐⭐ proper, connectors Saas Website

The ❓ indicates the information is to be completed
The ⚠️ that the functionality can be implemented either in a roundabout way or as a hack

Although these solutions share some features with Datami, they can differ significantly in terms of cost, complexity and specific features

It is important to take into account the specific needs of each project before choosing the most suitable online data visualization and editing solution

Technical roadmaps

Here is a list of different features
that reflect requests and needs
from our users to date.

This roadmap is to be taken as a whish list of features that we consider interesting and compatible with Datami, and that we would like to be able to develop if we manage to find the corresponding funding.

The features’ developments are described as follow :

  • Milestone : family of features
  • Features : feature to develop
  • Priority ( 🔴 ) : importance given to the feature by the users
  • Difficulty ( ▪️ ) : anticipated comlexity of the feature
  • Dev. + man. : sum of development and management (in days per person)

Technical roadmap 2023 - global


Classified by milestones

Milestones Priority Difficulty Dev + man.
More data sources 🔴 high ▪️ easy 38 days
Online widget configuration 🔴 high ▪️▪️▪️ hard 26 days
Manage contribution widget 🔴 high ▪️▪️▪️ hard 25 days
Protect widget with password 🟠 medium ▪️ easy 12 days
Better UX - data management 🟡 low ▪️▪️ medium 42 days
Better UX - data interaction 🟡 low ▪️▪️ medium 44 days
Better UX - maps 🟡 low ▪️▪️ medium 37 days
Better UI - customization 🟠 medium ▪️ easy 13 days
Better UI - new views 🟡 low ▪️▪️ medium 27 days
Refactoring 🟠 medium ▪️▪️▪️ hard 48 days
Tests 🟡 low ▪️▪️▪️ hard 18 days
TOTAL 330 days

Diagram of future developments

Elements in orange symbolize the main milestones to be developped as described in the roadmap above.

Elements colored in turquoise symbolize features already existing in Datami.

  • A : Online widget configuration
  • B : Manage contribution widget
  • C : More data sources
 

Technical roadmap 2023 - detailed 1/2

Milestones Features Priority Difficulty Dev + man.
More data sources Connect to external APIs - generic 🔴 high ▪️ easy 5 days
More data sources Connect to external APIs - OSM 🟡 low ▪️ easy 5 days
More data sources Activity pub integration 🟡 low ▪️▪️▪️ hard 18 days
More data sources Connector to SourceForge 🟡 low ▪️▪️▪️ hard 5 days
More data sources Connector to BitBucket 🟡 low ▪️▪️▪️ hard 5 days
Online widget configuration Interactive interface + preview 🔴 high ▪️▪️▪️ hard 20 days
Online widget configuration Save new config to git repo 🔴 high ▪️▪️ medium 6 days
Manage contribution widget Interface loading last PRs from repo 🔴 high ▪️▪️▪️ hard 15 days
Manage contribution widget Accept / moderate PR + messages 🔴 high ▪️ easy 10 days
Protect widget with password Protect access before showing widget 🟠 medium ▪️ easy 6 days
Protect widget with password Special token for protected widget 🟠 medium ▪️▪️ medium 6 days
Better UX - data management Add a new column + update schema 🟠 medium ▪️▪️ medium 10 days
Better UX - data management Drag & drop CSV to widget 🟡 low ▪️▪️ medium 10 days
Better UX - data management Save CSV file to Git repo 🟡 low ▪️▪️ medium 4 days
Better UX - data management Add / drag-drop a picture in a cell 🟡 low ▪️▪️ medium 6 days
Better UX - data management Cache user changes / branch until pushing 🟠 medium ▪️▪️▪️ hard 12 days
Better UX - data interaction Full screen - debug 🟠 medium ▪️▪️ medium 6 days
Better UX - data interaction Range filter 🟡 low ▪️ easy 4 days
Better UX - data interaction Change width column 🟠 medium ▪️▪️ medium 8 days
Better UX - data interaction Helper at loader 🟡 low ▪️ easy 2 days
Better UX - data interaction Better integration of Frictionless data packages 🟡 low ▪️▪️▪️ hard 12 days
Better UX - data interaction Export as pdf 🟡 low ▪️▪️▪️ hard 12 days
Better UX - maps Inject data to vector tiles on map view 🟡 low ▪️▪️ medium 8 days

Technical roadmap 2023 - detailed 2/2

Milestones Features Priority Difficulty Dev + man.
Better UX - maps Add or edit geojson objects 🟡 low ▪️▪️▪️ hard 29 days
Better UI - customization Custom styles / CSS / logos 🟠 medium ▪️ easy 4 days
Better UI - customization Accessibility 🟠 medium ▪️▪️ medium 9 days
Better UI - new views Agenda view 🟡 low ▪️ easy 8 days
Better UI - new views Graphs view with D3js 🟡 low ▪️▪️ medium 10 days
Better UI - new views Simultaneous dataviz + map on same view 🟡 low ▪️▪️ medium 9 days
Refactoring Put all git* requests into a package 🟡 low ▪️▪️▪️ hard 15 days
Refactoring Migration to Typescript 🟠 medium ▪️▪️▪️ hard 15 days
Refactoring Migration to Vue3 🟡 low ▪️▪️▪️ hard 18 days
Tests Add functional & unit tests 🟡 low ▪️▪️▪️ hard 18 days

Technical roadmap 2023 - global


Classified by priority / difficulty

Priority / Difficulty ▪️▪️▪️ hard ▪️▪️ medium ▪️ easy TOTAL
🔴 high 35 days 6 days 15 days 56 days
🟠 medium 27 days 39 days 10 days 76 days
🟡 low 132 days 47 days 19 days 198 days
TOTAL 194 days 92 days 44 days 330 days

Retroplanning

Note : this schedule is expected to evolve according to user requests and funding.

 

Credits

A project by the multi cooperative

Our cooperative contributes to the development of digital commons and associated services, by bringing together a community of professionals working for a digital of general interest

Our first users

 

Our sponsors

Datami was laureate of the Plan France Relance 2022 and has benefited from the support of the following organizations

 

Thanks for your attention !


 

Datami

is a project led by the cooperative

multi

 
 

contact@multi.coop