Superset allowed us to replace Tableau and not looking back

Took me a while figure out how to embed it into my app using Superset Embedded SDK.

Superset Embedded SDK - "Embedded SDK allows you to embed dashboards from Superset into your own app, using your app's authentication. Embedding is done by inserting an iframe, containing a Superset page, into the host application."

https://github.com/apache/superset/tree/master/superset-embe...

Superset is based on very high quality and well maintained chart library eChart

https://echarts.apache.org/examples/en/#chart-type-linesG

Community Roadmap

https://github.com/apache/superset/projects?query=is%3Aopen

Huge respect to Preset.io and its team for contributing to the project and keep it in a great shape

https://preset.io/blog/

Superset source code is very easy to read and understand, and as a result it's possible to implement some advanced caching techniques reduce the load on charts.

No BI is perfect.

Watching Superset for years gives me confidence the project will work as supposed down the road, and eventually some of its packages can be reusable for all kind of visualizations and data hacking.

Our main approach to visualisation is to start with eChart and simple Reactjs wrapping and spin off Superset on subdomain for power users, and later see which one works better. Same look gives a very pleasant experience.

Previous HN discussion: https://news.ycombinator.com/item?id=35645464 (97 comments)

Reminds me Obsidian DataView but with charts https://github.com/blacksmithgu/obsidian-dataview

This whole ideas to have data, visualisations and knowledge base in one private offline place is very appealing

The Markdown Markup typing experience is just so good compared to e.g. Slack, Reddit and other markdown-esque tools

All in all, not very innovative, but highly needed open source version of a traditional BI tool. Definitely something to follow and to use in temporary, not too demanding use cases. And hopefully a future replacement of Tableau or Power BI.

Their interactive "embedded-mode" avoids iframes too... but it's built with web components, so you wind up in shadow-DOM hell if you want to do anything dynamic on the view's contents.

https://www.tableau.com/products/server

If you have money, dedicated team of data analytics who are already familiar with Tableau - no need to torture them with other tools.

Superset is lightweight and open source, but only has 5% of the features. So it really depends what you need!

I tried Superset a few years back, and maybe it's changed since then, but intuitive is about the last thing I'd use to describe it. Things which I could figure out in a few minutes on any other BI tool literally took me hours of searching. It didn't help that they decided to rename core concepts at some point so half the online documentation made no sense anymore. Others at those companies who tried it at the time said similar things.

I like Grafana too, but there's basically no isolation between your query and the SQL database at least in the Altinity Grafana plugin for ClickHouse which is the main one I use.

Instead I installed Metabase in 5 minutes tops: spin ec2 instance, whether and java -jar . I've never looked back.

The only thing that turns me off I'd that it's implemented in an obscure language. At one time I wanted to add some custom postprocessing to an api (given an sql query, get some python/pandas postproc command from a sql comment and execute it in the returned table), but the used language is just not for me (some lisp dialect)

https://www.metabase.com/demo

Demo is nice.

1. https://www.apache.org/foundation/how-it-works/#incubator

1. Built-in data warehouse - We spin up a duckdb database for you to load data to

2. 500+ connectors - You don't need to buy a separate ETL and you can pull in all your data (e.g. Postgres, Stripe, HubSpot, Zendesk, etc.) automatically

3. Semantic layer - Define dimensions, measures, and joins in one place. We have pre-built models for all the sources we support (e.g. the Stripe model already has measures for MRR, churn, etc.)

4. Simple BI - Build a table with the data you want and generate visuals off that table

I'm [email protected] if you have any questions.

And the documentation is sparse at best.

What it was though, was riddled with dozens of Python runtime errors and innumerable glitches.

Metabase is where it’s at.

But they may be an overkill if your primary use case is to infrequently build semi-interactive reports for non-technical end-users and your use cases are are mostly covered by standard graphs & tables. Esp. so if you are familiar with SQL and have access to the underlying data source. Two nifty utilities I have found to be very useful for latter kind of use cases are SQLPage and Evidence.

They make it very convenient to whip out some SQL and convert that to a neat professional looking web ui that can be forwarded to an end user. In case of Evidence it is a statically generated site, and in case of SQLPage it is a web app that connects to a live database.

SQLPage: https://sql.ophir.dev/

Evidence: https://evidence.dev

https://superset.wmcloud.org

https://phabricator.wikimedia.org/T169452

Back then, I used this to generate some custom statistics

https://github.com/altilunium/wikiidmon

I've been running it in production since 2017, at two jobs, the current one a big corporation.

Best general-purpose, database-backed dashboarding system out there. I would never pay for Tableau or PowerBI.

Same for Airflow.

Open source Business intelligence platform made with Python - https://news.ycombinator.com/item?id=29368664 - Nov 2021 (49 comments)

Apache Superset 1.1 - https://news.ycombinator.com/item?id=27439939 - June 2021 (28 comments)

The Apache Software Foundation Announces Apache Superset as a Top-Level Project - https://news.ycombinator.com/item?id=25905277 - Jan 2021 (1 comment)

Apache Superset is an enterprise-ready business intelligence web application - https://news.ycombinator.com/item?id=21133931 - Oct 2019 (7 comments)

One thing that helps is hooking metabase up to its own database and building queries on your queries, e.g.:

    select *
    from report_card 
    where dataset_query ilike '%' || {{query}} || '%'

We also try really hard to keep the Collection directory structure clean and consistent. But it's still really hard.

Another option could be to use LLM to summarize, tag and group queries for better discoverability.

Is it worth it for BI on small datasets?

The reason we chose Metabase was that it had table joins, while Superset doesn't (unless it has added them since I used it). It also looks a bit sleeker. But I strongly prefer Superset; I found that with Metabase I had to turn a lot of things off to make it usable (Let me see "the_table" not "The Table"!), I was constantly annoyed at the opacity around models vs "questions", etc. and every time I wanted to change a question Metabase insisted on creating a new one instead. The real issue here was when we wanted to swap out the data source for a lot of questions but there was no clean way to do so without MB just creating new questions.

Also, Metabase doesn't have serialization unless you pay them AND you self-host, (if I'm self hosting then what exactly am I paying for?) and that's pretty annoying. https://www.metabase.com/docs/latest/installation-and-operat....

But it does let you join tables. Sometimes that's enough to make MB worth dealing with.

I've found the weird "make it easy" mindset a bit annoying with Metabase too. The whole questions, nice table names...

I'll give Superset a try in my next project I think.

Yeah, somewhere along the line Metabase decided to get opinionated on "self-serve". I imagine it works well for some teams and companies, but for the tech-oriented, it's annoying.

I prefer my BI tools to be platforms that make for easy charting and cross-filters, while I build and control the models behind the scenes with a tool like dbt.

Any recommendations for a good piece of software for the single user case? Or a more convenient way to run the heavyweight tools?

I see these products as tools for data visualization and reporting i.e. presenting prepared datasets to users in a visually appealing way. They aren't as well suited for serious analytics.

I can't comment on Superset or Tableau but I am familiar with Power BI (it has been rolled out across my org), the type of statistics you can do with it are fairly rudimentary. If you need to do any thing beyond summarizing (counts, averages, min, max etc). It is not particularly easy.

For data analysis I use SAS or R. This software allows you do things like multivariate regression, timeseries forecasting, PCA, Cluster analysis etc. There is also plotting capability.

Both these products are kind of old school, I've been using them since early 2000's, the "new school" seems to be Python. Pretty much all the recent data science people in my organization use Python. Particularly Pandas and libraries like Seaborn (https://seaborn.pydata.org/).

The "power" users of Power BI in my organization tend to be finance/HR people for use cases like drill down into cost figures or Interactively presenting KPI's and other headline figures to management things like that.

This is a single user application, unless you make it part of your built application.

K8s installation instructions: https://superset.apache.org/docs/installation/running-on-kub...

RBAC configuration: https://superset.apache.org/docs/security/#rest-api-for-user...

[1] https://blog.adnansiddiqi.me/create-your-first-sales-dashboa...

Many, or most, users for a BI tool will be operations, product managers, and business management who simply will not find the interface to be intuitive, responsive, or well designed. At least that's my experience.

It's part of our Open Source Data Platform and it's one of the few open source BI tools out there and there are not a lot of alternatives in this space. We generally like it.

Apparently installation will not work with Python 3.12, dur to deprecation of distutils.

Does anyone have any method to install this?

https://superset.apache.org/docs/installation/installing-sup...

Wes McKinney used to have an excellent 5 minute introduction to pandas in this genre.

Superset Embedded SDK - "Embedded SDK allows you to embed dashboards from Superset into your own app, using your app's authentication. Embedding is done by inserting an iframe, containing a Superset page, into the host application."

https://preset.io/blog/accessing-apis-with-superset/

"Shillelagh (ʃɪˈleɪlɪ) is a Python library and CLI that allows you to query many resources (APIs, files, in memory objects) using SQL. It's both user and developer friendly, making it trivial to access resources and easy to add support for new ones"

https://github.com/betodealmeida/shillelagh

we have multiple tenants + developer instances of our warehouse. to reuse the same dashboard in this setup we need to create at least 3 virtual datasets, plus wrangle a bunch of boiler plate jinja.

In what way, any details?

Not been tracking that or using OpenOffice for a while.

If you want to style the whole application, you can fork the repo and go bananas. If you're looking for theming, there's more to be done yet on that front, and I wrote an article on that too: https://preset.io/blog/theming-superset-progress-update/

To make it more concrete -- coworkers tell me Grafana doesn't work so well with Apache Druid, while Superset supports it quite well.

superset is the flip side of grafana; not good for up-to-second updates, but good for complex queries. Also, non-time series stuff. Ex: Which customer groups bought which products for all time?

As such, closer to an open source replacement for PowerBI.

https://www.youtube.com/watch?v=RY0SSvSUkMA

https://github.com/apache/superset/discussions/20094

I think Athena can only query data on S3?

I wrote about Superset's semantic layer here: https://preset.io/blog/understanding-superset-semantic-layer...

One popular option is to use dbt or Cube for the semantic layer and pair with Superset: https://preset.io/blog/announcing-presets-ui-integration-wit... and https://preset.io/blog/open-source-looker-cube-superset/

I built my own semantic layer instead. I use this in production in my company but obviously use at your own risk as it's a one-man show.

https://github.com/totalhack/zillion

So now that I picked Metabase, Superset is topping HN for no apparent reason. Why?

https://preset.io/

I don't think it's semi-abandoned. I had a brief interaction with the project in my previous job, and I found the community and the company to be reasonably engaged and responsive.

Hmm. I suppose all open source looks that way if it doesn't get regular funding/attention.

Apache does house a lot of abandonware. They had some relevance as recently as 6-7 years ago but they've been largely replaced by nginx I think. That being said, I view them like the local soup-kitchen - important to have and maintain, but not where I want to go for a 5-star meal.

These projects don't give the apache foundation an appearance of importance or relevance, rather they make it look rather rundown.

Obivously the main "alternative" is for the original company to simply shut down the product/service, which can do irreperable harm to a company when they have high-profile customers who are utterly dependent on a service.

Another alternative is to use an open-source foundation that's directly managed by the original company, which is what Microsoft did with its DotNet Foundation ( https://dotnetfoundation.org/ ) - and while Microsoft's legal team ensures the foundation is "legally" independent, in practice we know all the significant shots are being called from within Microsoft-proper; but it does give us some modest reassurances that .NET won't suddenly return to being closed-source overnight.

Another alternative is to not open-source it and to instead sell it off to another company that can maintain it while still being profitable - this is what Adobe did with Flash: they sold it all off to Samsung because their Harman division wanted to continue using Flash for embedded/automotive UX work. This approach can work, but doesn't benefit the wider ecosystem the way that open-sourcing does - and something something shareholder value and return-on-investment by selling rather than writing-it-off...

What companies won't do is let any of their engs that are passionate about a project split-off from the company to run and maintain it, le sigh.

I think the HN algo is pretty easily manipulated. I worked at a startup that had an effective process to get things to the front page

That sounds (potentially) sleazy. If you think it's a technique that HN could potentially defend against, I encourage you to explain it to [email protected].

Pretty sure it's as simple as posting in your general slack channel "@here we posted a new article to HN, go upvote and write a comment"

Business users loved the self-serve query builder, and it wasn't uncommon to walk around the office and see Metabase up on someones screen. My CEO absolutely loved it, and used it daily including to put together data for board decks.

None of my users cared about visualizations, and lived in tabular data. This included finance, marketing, merchandising, operations, and executives (CEO/COO/CFO). The only people that lamented the limited visualization were analysts. Power users did all their day-to-day work in Excel or other tools anyway, such as managing marketing spend or inventory allocations.

Metabase was great for dashboards and self-service (ad-hoc). 10/10 would deploy again.

Can you name a few examples?

https://projects.apache.org/projects.html?name

These are all projects that once were (more) relevant, however seem to have become rather niche (Gradle, Jetbrains/VSCode, GoogleDocs/Libreoffice e.g. for the first three are the dominant competitors).

Most of these projects (like the massive commons listings) are either used by some Java library somewhere (meaning their success/relevance is tied to the usage of Java), or are obscure enough that they are no longer used widely and so suffer from lack of interest.

There are gems in this list, to be sure, but if you just run into half-maintained projects all the time you're not likely to associate good things with the Apache name?

OpenOffice is probably the most famous (it still has the name, but it is dead, LibreOffice is the real "active" fork).

And the things in the "Attic" are officially dead - https://projects.apache.org/committee.html?attic and many more projects should be there.

Lots of other projects just die silently and/or you are unsure of the status.

Here you at least have a chance to revive them if you like as there is always an overarching organisation.

More things should move into the Attic, like OpenOffice.

Amen brother.