I want to love Terraform but it's such a horrible platform to code on:
* Error messages are overly verbose yet cryptic, and sometimes even unrelated to the actual error raised by the cloud provider themselves. Coupled with the lack of line numbering or other helpful identifier aside the unnecessarily long module hierarchy and debugging those scripts is a massive exercise in frustration and usually far more time wasted than really should ever be necessary.
* HCL is a hateful "language". The fact you cannot order stuff procedurally means you're constantly running into dependency issues on larger deployments. And dont even get me started on the "count" kludge to work around a lack of proper iteration.
* There is a lack of internal consistency with the support of different methods. Eg "count" does t always work with all resource types. Some resources cannot have properties defined with variables.
* Using calling modules requires so much bootstrapping code. It's just painful.
I get Terraform is the best we have for multi-provider deployments but their idea to create a superset of JSON only to then compile that back down to JSON anyway was such a poor decision in my personal opinion. I get the point was to have something that was accessable to non-programmers while still expressive enough for developers to use; however instead what they've created is a monstrous language that is too complex for the former group and too irrational for the latter.
I've been been very tempted to write my own Terraform alternative based on my experiences using it (and CloudFormation) - I even already have another programming language that Ive written a parser for and would be well suited for this type of application. But my time is pretty limited at the moment so I struggle on Terraform.
Funny thing is, HCL isn't actually a superset of JSON. For example,
{
"foo": { "bar": 1 }
}
can't be represented in HCL. (Even Consul had to add a kludgy hack to support HCL config as a result.) Instead, they call HCL "JSON-compatible", which I think means JSON can be written to represent any equivalent HCL structure (HCL is a subset, essentially).
That said, you might be interested in Terraform 0.12 [0], which will be using some new HCL v2 that actually has first-class expressions and dynamic blocks (for loops). And, finally, a ternary operator that short-circuits. Unfortunately, the dynamic block stuff looks like it's based around for-loops and doesn't support just regular if-statements... but we'll see where that goes.
Thanks for the link. Ive not yet had a chance to play with Terraform 0.12 but from what I've read it does sound like HCL v2 is definitely a step in the right direction. However so long as it's primarily a data serialisation format I think I'm going to take issue writing Terraform code in it because sometime you just need to express something procedurally. Maybe I'm just in the minority here? Or maybe I've just been spoilt with tools like Puppet and Bash but I can't help feeling that HCL is a step backwards in terms of expressiveness.
I wasn't aware about the JSON subset / Consul problem though. That's really interesting to read. It's funny because back when I was building test Consul cluster I did wonder why JSON was used for config instead of HCL. I guess now I know why.
I agree with you 100%. I'm pretty excited about HCL v2 - already simple stuff like a short-circuiting ternary operator makes my life easier (no more weird joins/splats with conditional resources in outputs). Hopefully further improvements are implemented on top of the 0.12 changes.
Otherwise, about like you, I'm tempted to write a Terraform frontend that interfaces with existing providers...
A lot of problems similar to your complaints exist for CloudFormation as well. Instead of complaining about the language warts, you’ll be forced to use an even more cumbersome set of primitives to access arrays and hashes, and the messages will only get less sensible as you look for either YAML whitespace errors or inevitably write a converter to avoid using JSON. CloudFormation limits like number of parameters and outputs start to be a real pain to scale to support a production environment beyond simple demo stacks, and Terraform has more issues scaling with team members concurrently trying to modify system state due to its less strict modularization / locking model.
Infrastructure as code tooling is all very primitive compared to what we take for granted writing most other traditional software but it will take some time and maybe another generation to do it well.
I don't know where you got the impression I was arguing that CloudFormation is better than Terraform from but I assure you that wasn't the arguement I was making. In fact I also made the same points you made in reply to another commenter in this discussion.
I know Terraform is the best tool we currently have (I even explicitly stated that I'm my previous post) but that doesnt mean there isnt still a massive room for improvement. Starting with the depreciation of HCL, in my personal opinion.
You might want to give cfn-builder[0] a try. I'm biased because I wrote it but I find that it's a good way to write and maintain my CFN templates. Also it's written in simple Nodejs and is easy to expand for your own needs.
CloudFormation isn't really good enough because it's AWS specific and doesn't track changes to the state. Plus it just uses data serialisation formats as well so doesn't even address the core problems I raised with HCL. What I really want is Puppet for Infrastructure; the closest I've used to that is Terraform but the syntax isn't quite there...yet.
Plus the pace of development in nodejs worries me. All too often I've ran into issues where modules have changed and broken things downstream. When you're running infrastructure as code you really want to be damn sure your tooling is going to be consistent for years to come and I really don't have that faith in nodejs. Sure, if your a JavaScript developer you can manage it easily enough, but if you're DevOps who rarely touches JS then you really want your tools to be low maintenance. So to that end I wouldn't consider any nodejs projects for any serious production work given the kind of customers I work for (high availability stuff for some major brands). I might be fine but it's just not worth the risk.
* Error messages are overly verbose yet cryptic, and sometimes even unrelated to the actual error raised by the cloud provider themselves. Coupled with the lack of line numbering or other helpful identifier aside the unnecessarily long module hierarchy and debugging those scripts is a massive exercise in frustration and usually far more time wasted than really should ever be necessary.
* HCL is a hateful "language". The fact you cannot order stuff procedurally means you're constantly running into dependency issues on larger deployments. And dont even get me started on the "count" kludge to work around a lack of proper iteration.
* There is a lack of internal consistency with the support of different methods. Eg "count" does t always work with all resource types. Some resources cannot have properties defined with variables.
* Using calling modules requires so much bootstrapping code. It's just painful.
I get Terraform is the best we have for multi-provider deployments but their idea to create a superset of JSON only to then compile that back down to JSON anyway was such a poor decision in my personal opinion. I get the point was to have something that was accessable to non-programmers while still expressive enough for developers to use; however instead what they've created is a monstrous language that is too complex for the former group and too irrational for the latter.
I've been been very tempted to write my own Terraform alternative based on my experiences using it (and CloudFormation) - I even already have another programming language that Ive written a parser for and would be well suited for this type of application. But my time is pretty limited at the moment so I struggle on Terraform.