So this is a very relevant question and I have been trying to figure out if I need to migrate off timescaledb asap coincidentally over the last 2 weeks (We're pre-production anyway, so it's the time to do so!). Doing so has been super low priority, but since you're here .... :)
If I have a table that records timeseries data and then another table that has a customer-provided extensible set of metadata where a customer can define columns and other related tabular data, would that violate the license? The customer doesn't have a direct, like, psql level of access but the API intentionally provides a very similar level of interaction.
Does this qualify as providing Data Definition Interfaces? If none of those additional columns and such appear on tables set up as timescale tables does that make any difference?
One clarification to the above discussion -- about whether these restrictions also get applied to "even the regular relational stuff" -- which I think is relevant to you (and parent):
The Timescale License only covers the TimescaleDB code. Postgres code continues to be covered by the OSS PostgreSQL License [0].
So putting aside the question about API vs. psql level (again, the Timescale License was drafted to enabled this for "Value Added Services", e.g., where "such value-added products or services are not primarily database storage or operations products or services"), this license wouldn't apply for non-Timescale code.
I appreciate from your POV this probably is silly but for us it’s very helpful to have explicit clarity since investors are questioning and demand certainty. I once had to rewrite https handshakes because a lawyer thought export compliance laws would be violated, so hopefully you understand my trepidation :-)
Not the parent, but great to hear that clarification! By the way, doubt I would complain if Prometheus didn't look impressive. :-)
One other bug report on the license language front, this language could be construed to prohibit uses like Prometheus--unless there's a definition of operations products I missed (possible).
> are not primarily database storage or operations products
Suggested edit:
> are not primarily database storage or *database operations products*
Oh sorry, said Prometheus above, meant TimeScale is impressive!
Understood. If you see the other thread regarding the "such as" language, there is a serious edit that you can batch-up and repair to reflect your intentions with this one.
The "such as" language, retains your right to sue anyone for license violations if their API allows any customer action that causes structure changes indirectly, via the DDL, even under the hood (materialized views, too, presumably). That's way way more use cases than just repackaging TSDB as a service. That's a landmine, which when people compare and choose databases, they'd just assume avoid, even if otherwise comfortable with a cloud-protective license. Making this clearer and less onerous probably will probably pay for itself with a wider top-of-funnel for the product with more people more confident in the license.
The "We Clarified It In a Thread on Hacker News Public License" is probably not as ideal as updating the places that need clarification. :-P
Yeah - I used to lead a department that would process somewhere around 10TB of CSV formatted data per day.
The edge cases are a hassle but they don't become less of a hassle from a business perspective by switching to json or really any other format. We tried an experiment of using more json and eventually gave it up because it wasn't saving any time at a holistic level because the "data schema" conversations massively dominated the entirety of the development and testing time.
Obviously being able to jam out some json helped quite a bit initially, but then on the QA side we started to run in to problems with tooling not really being designed to handle massive json files. Basically, when something was invalid (such as the first time we encountered an invalid quote) it was not enjoyable to figure out where that was in a 15GB file.
That said, I fully concur with the general premise that CSV doesn't let you encode the solutions to these problems, which really really sucks. But, to solve that, we would output to a more columnar storage format like Parquet or something. This would let us fully encode and manage the data how we wanted while letting our clients continue working their processes.
What I would really like to see is a file format where the validity of the file could be established by only using the header. E.g. I could validate that all the values in a specific column were integers without having to read them all.
Really appreciate the insight from you and the GP here. I have been struggling with data format decisions around a personal project that will only be used by a few people, being unsure about the extent i should try to make it bulletproof (but harder to maintain and modify) or just keeping it simple (but primitive). It's helpful to see an experienced professional perspective showing that you can fall into a tooling rabbit hole at any scale.
> "data schema" conversations massively dominated the entirety of the development and testing time.
Agreed. JSON let's me know something is a number. That's great, but I still have to check for min/max,zero etc. A string? That's great, but I got to check it against a set of enums, and so forth. Basically, the "types" JSON gives you is about 20% of the work, and you're going to have to parse things into your own types anyway.
> What I would really like to see is a file format where the validity of the file could be established by only using the header.
Are you saying something like a checksum so not only is a schema provided but some method to verify that the data obeys the schema?
If you're talking about just some stronger shared ontology, I think that's a direction things will go. I call this concept "Type the world" or "World Wide Types". I'm starting to think something like GPT-N will be the primary author, rather than a committee of humans like Schema.org.
Honestly with the schema thing I'd probably be fine with either/or!
A checksum would be crude and user-hostile, only being able to say "you did it wrong" but not really good at tell you what it means to do it right.
If I understand the concepts correctly then it seems like a shared ontology could potentially solve the problem in a non-hostile way.
Plus, it makes me happy because I feel like types are a real-world problem, so it is always nice if the type system could enforce that real-world-ness and all the messiness that comes along for the ride.
We looked at it and there were a few problems we had with where it would force us to put VMs that we just weren't super comfortable with due to the in-process-ness.
More a byproduct of decisions made 5 - 7 years ago when the company was in raw startup mode versus a more mature roadmap.
On the flip side, the only time I've ever felt like my work was fulfilling and valuable was when was under-compensated and insanely overworked (like 80 hours was a pretty reasonable work week).
The "Tiny town" is 70,000 people who live in an multi-city metro area of around 600,000. That may not be like SF or NYC big, but it's also not just a random metro area. Having lived there for a few years, you're painting with a poetic brush a bit and I'd like to clarify how I see it.
Walmart owns Bentonville. The city council, mayor, and basically anyone there is almost completely subservient to their interests. Again - WM never moved in to the small community, the small community literally is Wal-mart.
For many years, people in the town worked in Wal-mart retail buildings & warehouses around Bentonville. Around 2013 the company started to struggle to compete and found that they needed to attract talent, so the Walton Family Foundation started spending somewhere around $300 million - $400 million a year in improvements and restoration projects. This included massive subsidies for companies that moved in to the area - WFF also paid partially or fully relocation money, but I have no idea how much they paid other than one restaurant owner I spoke to who was basically given moving costs, a building, and "free" rent for a year.
The whole "Wal-mart HQ" thing is just another in their line of tactics to attract new talent in to the area because they are struggling to build an ecosystem of talented developers and analysts. Which, again, there's nothing wrong with this tactic if that's what you're hunting for.
One thing I'll give WM for sure - they are 100% focused on building out a tech bubble for people who want a semi-urban lifestyle. There's a lot of TX & CA transplants and honestly it's working great for them.
All told, none of this is even kind of comparable to Amazon moving to NYC or Atlanta. Nor is it comparable to a company moving in to a small town brand-new.
According to wikipedia the population of Bentonville when the first WM store was built was 2,949.
It's now 54,909, but maybe it's 70k as you say.
Either way it sounds precisely like a situation where a giant corporation moved in to a small town. Or more accurately the tiny store grew into a giant corporation and so did the small town.
My point was merely that many small rural town mayors would love it if a giant corporation would move an HQ with tens of thousands of high paying jobs into their tiny communities! This is why the are willing to give giant incentives to them.
You make great points otherwise, thanks for sharing your perspective!
I've worked with several recruiters who are trying to shop around as an outsourced recruiter. Basically, they take your resume and slap their name on it in an effort to show companies "You should use me because I provide well-qualified talent."
If you're not going through internal recruitment then this is a great way to go about it. The recruiter isn't just trying to get their 20%, so they spend a lot more time getting to know you and also getting to know where you're going.
The biggest down side is that you'll never hear a negative word about anything, so you gotta be good at asking some pointed questions.
I work in this industry as well. That said - private label is a small (but growing) area in the US markets, so it's hard to make a generalized statement of how much someone does or does not participate.
Many retailers are beginning tie production of private labeled products in with being the captain of a category - which begins to create incentive for companies to start to pursue these private label opportunities.
K-C and P&G are two examples of companies who largely resist the private label trends in the US - you could counter with ConAgra and Treehouse.
A lot of that has to do with the product and what-not, of course.
To be honest a lot of your suggestions are probably very solid but come off like the "Find a mechanic you can trust" solution.
By that I mean the advice of "find a mechanic you can trust" only is applicable to someone already familiar enough to know how to determine if a mechanic is trustworthy.
A lot of your advice is probably 100% valuable, but as someone working through a lot of these same concerns I'm stuck wondering "Well, how do I know I'm actually doing that?" It leads me to wonder if I'm actually successful at any of this or just Dunning–Kruger-ing my way through it.
Our mechanic was ripping us off. There's this scam where they overtighten the screws on the oil pan and crack the gasket. Oil leaks out very slowly and in some cars ends up on the leads to the battery. This corrodes the leads. They then charge you for a new battery and tell you that there is a electrical problem in your car that will cost thousands of dollars, etc, etc. Luckily I knew about this scam and when our car started losing oil directly after an oil change and the mechanics refused to do anything about it, I was able to tell my wife what was about to happen. This convinced her that the mechanics were scamming us.
So the question became, who do we send the car to. There was a young guy down the street. He was taking apart cars and putting them back together again. His friends were all into racing and while he had previously worked at a dealership, he quit to live the dream of repairing racing cars. Of course he was broke. I told my wife to wander over and see if he wouldn't mind looking after our car. Great mechanic and saved us thousands compared to what the swindlers were doing. Previously the car was breaking down all the time. After we switched it never had a problem.
When we replaced our car recently, we gave him our old car as a thank you. Good mechanics are worth keeping happy :-)
It's not really a science for finding people who are good at what they do. There is not a thing that you can pinpoint and say, "That person is definitely good and that person is not". But I think if you start with the proxy of, "That person loves what they do and sacrifices a lot in order to do it", I think you are more likely to find someone good. Not 100%, but you've got a much better shot at it.
Edit: I misinterpreted your post and assumed you meant that you were looking for a good lieutenant to help you. I need more sleep! But either way, work at being that good mechanic in the same way -- love what you do and work hard at it. You'll definitely make progress, so don't worry about it.
I'm really glad you were able to avoid that scam. That's super scummy.
I built this premise because I do read a lot of car forums and, quite frequently, people would ask "Hey my car is doing XYZ" and a lot of responses would be "Take your car to a mechanic you can trust." To me this is basically telling them "Already know the answer to your question."
One time a guy answered basically saying "Hey, go find a guy who has a car in his driveway he works on every weekend, bring a 6 pack, and ask him for an opinion." It totally opened my eyes to how different types of advice really change people's perspective.
I believe that the value is in explaining what a person who loves what they do actually looks like.
I was recently recommended a couple books called "Extreme Ownership" and "The Dichotomy of Leadership". They tackle the idea of how to be a leader and what that means in the day to day.
They break down the communication with your team into a principle you want to strive for. Then you're shown an example of how that happens in the real world.
Once you've seen the patterns, you notice them everywhere, and you begin to unravel the issues, one hurdle at a time. It's like seeing a great design pattern, seeing where it fits and how to apply it, except now you're doing it with communication instead of code.
I think this might help you "find a mechanic you can trust" and get to the heart of what it takes to lead your team.
A struggle I've had is I'm used to casing performance in terms of errors or negative feedback. So far the feedback I've had is almost all positive (team is far above previous performance, flight risk is far lower than other departments, etc) but I don't feel like that's helping me improve cuz I don't know how to actually deal with positive feedback.
Some personal introspective criticism I've been wrestling with.
I've been working on this for a product launch we are trying to do and have had the problem you are talking about.
The problem is that explaining that without ending up mired in detail is much harder than people give it credit. I've found it very useful to actually write out the list of what I am trying to convey and look at it constantly while doing design work.
Also splitting out the work - the first forays have been what I realized was actually development work that I was hiding behind design. During design I put away the dev environment and just work with sketches.
You nailed why I think many object oriented designs fall flat. People presuppose both that the objects within their domain encompass all objects (just students and courses) as well as that the objects within a domain will not change.
When #1 is missed I usually see that theres a design that doesn't mimic its domain and thus lose the ability for developers and users to have clear, concise communication. At that point OO is a disservice.
When #2 is missed you end up with IFilteredCourseAdapterProcessor as people attempt to bolt on components to solve future needs.
The addition of the "Registrar" to the domain immediately demonstrates how the naive interpretation is missing core components and I bet the users and devs fundamentally aren't speaking the same language.
This, imo, leads to conversations like, "Of course so and so approves all the registrations! Otherwise it would be madness!"
If I have a table that records timeseries data and then another table that has a customer-provided extensible set of metadata where a customer can define columns and other related tabular data, would that violate the license? The customer doesn't have a direct, like, psql level of access but the API intentionally provides a very similar level of interaction.
Does this qualify as providing Data Definition Interfaces? If none of those additional columns and such appear on tables set up as timescale tables does that make any difference?