@akalitenya are you even in the clear with this? Some of the Old LookML syntax is an exact copy.
But more importantly, the challenge for any such tool is to go beyond use by 2-3 people. At 2-3 people anything will work. Where BI tools (open source and close source) struggle is scale: having all the right features for, essentially, a group of users who actually don't know how to work with data (did I just say that aloud?). Chartio caps at 20 people. RJ capped at 50-100 (and later became Stitch for that reason). We haven't seen where Metabase caps, but I bet it is in a similar range. Very few BI products have actually surpassed 100 users at target installations. And beyond 1,000 is a real challenge that only few, and even then with a lot of assistance, can support: Tableau, Looker, Microstrategy, maybe Birst, maybe Domo.
Also, a combination of BI with LookML is a complicated product. During my days at Looker, we were handling 50+ bugs / week, and filing 1,000+ tickets. Every day we were filing over a 100 new features.
So with all that, the question is, is it really worth the struggle? What's the end vision for supporting this? Why should someone who implements BI for a living bet on this product?
I met the online Looker demo a few years ago when I was looking for a business intelligence tool for another project.
Looker has a closed source code, so I did not see what algorithm is used to build queries.
I kept those LookML features that I understood and liked. However, in some places LookML is confusing (for example - references and aliases). I made them differently.
Later Looker quit using YAML.
> Very few BI products have actually surpassed 100 users at target installations. And beyond 1,000 is a real challenge that only few, and even then with a lot of assistance, can support: Tableau, Looker, Microstrategy, maybe Birst, maybe Domo.
Scaling that size is not a top priority right now.
> Also, a combination of BI with LookML is a complicated product.
You should have told me this a couple of years ago. It seemed pretty simple.
> So with all that, the question is, is it really worth the struggle?
I used Looker for years and have always wanted a product priced less for the enterprise and more for developers. I even prototyped a similar tool myself.
Congratulations on the launch. I will be using mprove and will give you feedback along the way.
EDIT: I would love to get in touch with you and learn more about what your plans are. Possibly collaborate if you'd be open to that. I'm dpaola2@gmail.com -- I couldn't find your contact info anywhere!
> Very few BI products have actually surpassed 100 users at target installations. And beyond 1,000 is a real challenge that only few, and even then with a lot of assistance, can support: Tableau, Looker, Microstrategy, maybe Birst, maybe Domo.
At one point Looker was essentially a web application with a query service that could be scaled by adding more servers behind a load balancer. Each end user is essentially working in a shared nothing environment and the query engine is driven by metadata stored in a git repository. Looker itself did not manage cache or analytical processes. All of the real effort to scale was in the database backend.
I agree with pretty much everything you've said. However, as someone who has used Chartio extensively, I'd say 20 is wayyy too low. It can definitely handle 100s. But, like you said, 1,000s is a struggle for anyone.
Also, if you think this is a rip off of LookML, you should take a look at what GitLab is doing with Meltano. They completely jacked LookML.
Hey @segah, founder of Chartio here. I can deeply second all the comments on how long of a feature tail BI is, and the amount of work it takes to have real product depth and stability vs. an impressive demo. It takes years, and ongoing maintenance that is impossible to estimate in the beginning.
Also, it's definitely a challenge to support 100's and 1000's of users digesting data, especially in the democratized fashion that we're typically used in. It takes good data governance, support, and admin tools. I gotta chime in and say for the record though that Chartio well supports many such customers.
It may be weird for you to chat as you worked at Looker, but I'd love to hear anytime on why you see Chartio capping out at a lower # than the others listed. You can reach me at dave-at-chartio.com!
I don't think ripping off LookML is a bad thing; on the contrary, it's probably an awesome thing for Looker to be in the position of shaping how companies model their data and the relationships thereof. At the very least, it means Looker can turn around and say "hey, if you're outgrowing this smaller tool that uses LookML, why not switch to a platform that can meet your scale (like, say, us)?". The more smaller-scale "competitors" aping LookML, the more teams outgrowing those platforms and looking for larger-scale platforms where they can `git pull $old-platform`, `git push $new-platform`, copy over some data sources / credentials, and call it a day. If $new-platform=Looker, then congrats, Looker now has a decently-sized customer.
In other words: Looker ought to embrace the idea of LookML being the data modelling language. Encourage other implementations of it. Treat them as customer sources instead of customer sinks. That's how they can take over the world :)
Why discourage an alternative? We've already seen there is demand for libre analytics tools as an alternative to google analytics. I'd use this product in the lower range as well <100 person company. I say go for it, looker is doing fine they won't care or be phased by this at all.
By support do you mean performance/load or array of features/team tools to be effective, support different use cases? Curious what you mean by 20 user cap for Chartio.
LookML supports different databases as sources. Sometimes Looker adds its own LookML parameters for database - which makes it difficult to read the documentation.
For analysts who work with SQL queries, any wrapper on a language seems complicated. But in the case of LookML and BlockML, in a couple of days you already begin to understand how this works, and after a week you can not tear yourself away from using the product.
Looker and Mprove are in the middle between fully custom DIY solutions and simple query generators tools.
The headlining feature seems to be that it has a dark and a light theme. This isn’t very encouraging for the rest of the product. I’m sure there’s a lot of good stuff here, but themes aren’t important enough to be the first feature mentioned.
Readit News
But more importantly, the challenge for any such tool is to go beyond use by 2-3 people. At 2-3 people anything will work. Where BI tools (open source and close source) struggle is scale: having all the right features for, essentially, a group of users who actually don't know how to work with data (did I just say that aloud?). Chartio caps at 20 people. RJ capped at 50-100 (and later became Stitch for that reason). We haven't seen where Metabase caps, but I bet it is in a similar range. Very few BI products have actually surpassed 100 users at target installations. And beyond 1,000 is a real challenge that only few, and even then with a lot of assistance, can support: Tableau, Looker, Microstrategy, maybe Birst, maybe Domo.
Also, a combination of BI with LookML is a complicated product. During my days at Looker, we were handling 50+ bugs / week, and filing 1,000+ tickets. Every day we were filing over a 100 new features.
So with all that, the question is, is it really worth the struggle? What's the end vision for supporting this? Why should someone who implements BI for a living bet on this product?
I met the online Looker demo a few years ago when I was looking for a business intelligence tool for another project.
Looker has a closed source code, so I did not see what algorithm is used to build queries.
I kept those LookML features that I understood and liked. However, in some places LookML is confusing (for example - references and aliases). I made them differently.
Later Looker quit using YAML.
> Very few BI products have actually surpassed 100 users at target installations. And beyond 1,000 is a real challenge that only few, and even then with a lot of assistance, can support: Tableau, Looker, Microstrategy, maybe Birst, maybe Domo.
Scaling that size is not a top priority right now.
> Also, a combination of BI with LookML is a complicated product.
You should have told me this a couple of years ago. It seemed pretty simple.
> So with all that, the question is, is it really worth the struggle?
Yes, If people will use it.
> What's the end vision for supporting this?
In future - maybe "skinny" or "thin" option mentioned here - https://medium.com/open-consensus/2-open-core-definition-exa...
>Why should someone who implements BI for a living bet on this product?
If you can not afford Looker (like me) and want to use similar product.
Congratulations on the launch. I will be using mprove and will give you feedback along the way.
EDIT: I would love to get in touch with you and learn more about what your plans are. Possibly collaborate if you'd be open to that. I'm dpaola2@gmail.com -- I couldn't find your contact info anywhere!
At one point Looker was essentially a web application with a query service that could be scaled by adding more servers behind a load balancer. Each end user is essentially working in a shared nothing environment and the query engine is driven by metadata stored in a git repository. Looker itself did not manage cache or analytical processes. All of the real effort to scale was in the database backend.
Also, if you think this is a rip off of LookML, you should take a look at what GitLab is doing with Meltano. They completely jacked LookML.
Is it something technically related or more business/feature related?
Also, it's definitely a challenge to support 100's and 1000's of users digesting data, especially in the democratized fashion that we're typically used in. It takes good data governance, support, and admin tools. I gotta chime in and say for the record though that Chartio well supports many such customers.
It may be weird for you to chat as you worked at Looker, but I'd love to hear anytime on why you see Chartio capping out at a lower # than the others listed. You can reach me at dave-at-chartio.com!
In other words: Looker ought to embrace the idea of LookML being the data modelling language. Encourage other implementations of it. Treat them as customer sources instead of customer sinks. That's how they can take over the world :)
Deleted Comment
For analysts who work with SQL queries, any wrapper on a language seems complicated. But in the case of LookML and BlockML, in a couple of days you already begin to understand how this works, and after a week you can not tear yourself away from using the product.
Looker and Mprove are in the middle between fully custom DIY solutions and simple query generators tools.
Quick point however - why do you need a new database ? You can use a table inside bigquery itself. It seriously reduces the dependencies required.
But you can't use BigQuery for OLTP.
https://github.com/mprove-io/mprove/blob/master/deploy/docke...
Can you not use biquery database itself. Create tables for your internal use instead of MySQL ?
Here's a recently built Graphite connector as well:
https://twitter.com/vadimska/status/1112816503055843330
/disclaimer: work in google cloud
I can't seem to find any on the site...
https://github.com/mprove-io/mprove