The part of this (really excellent) writeup that caught my attention was this:
someone writing a bootloader or firmware can stick to using Swift structs and have a simple guarantee of no dynamic overhead or runtime dependence
There seems to be a pretty strong implication here that Swift could be used to write firmware/bootloaders, and other low level constructs - including operating systems. Has anyone worked with Swift yet on that type of project?
The performance variations in Swift are (as of now) much larger and less predictable than the overhead of dynamic messaging in Objective-C, and the latter can always be removed by using IMP-caching or converting to a C function.
The claim is not about Swift in general, but of a limited subset of the language: "someone writing a boot loader or firmware can stick to using Swift structs and have a simple guarantee of no dynamic overhead or runtime dependence"
Based on what I know about Swift (reading Apple's manual, a tiny bit of experimenting, and decent knowledge about how compilers work), I would expect Swift to compile purely statically, like C, for this case. Performance may still be lower because the compiler is newer, but I wouldn't expect performance variations.
And of course, the standard library may not be optimal for this use case. The implementation of strings, for example, may be too dynamic for embedded work.
if you're going to limit yourself to a subset of a language, then any "language" can offer that guarantee. but it's really not that language any longer
My measurements were also with a limited subsets: integers, floats and arrays.
Depending on compiler optimization settings, you get from 30% to 1000x (yes, 3 orders of magnitude!) slower than C. Yes, technically that's not "dynamic" overhead, but from where I stand that's still orders of magnitude more variability and unpredictability (for reasonable values of "predictability") than even byte-coded Smalltalk.
Unbounded integers are much slower than wrap-around integers that C uses. So, unless you are comparing some kind of Swift int64 type with C ints, you are comparing apples to oranges. The reason is that for every "x + 1" expression, Swift has to ensure that the value doesn't overflow the size of a hardware register. C of course has no such qualms.
I don't know. Apparently integer arithmetic can cause overflow exceptions which I assume requires the runtime environment which would be outside of the restricted subset. I don't think Swift can be used to profitably write a bootloader but we'll see.
The operation + on integer types maps to a built in method which performs addition with overflow checks. At the assembly level this looks like an "add" followed by a "jo" (jump on overflow set) to an undefined exception such as UD2.
If you want wraparound addition you can use +& which removes the overflow check.
I wonder why the designers of Swift decided to make arrays structs on the outside but complex on the inside. Why not make them objects, i.e. reference types, as in Java and .NET?
I've seen increasing levels of unpredictability at a surface level with all languages as compilers have improved. The difference between a constant expression being folded or not can be easily of the order of 1000x (once it has been propagated through loops etc.). If you've still got the source for your tests then it's well worth looking at the IR emitted and the assembly produced from that to see where those differences are coming from.
I'd be very interested to know what that benchmark is, because you're either doing something very wrong, or it isn't relevant to the systems programming topic being discussed. A 1000x perf difference should be easy to show.
Just summing floats in an array, compiled without optimization, and yes, that surprised me.
My Postscript interpreter written in Objective-C was 2x faster. :-)
With optimization that went down to around 10x and wouldn't budge, even with -Ounchecked, UnsafeMutablePointer, varying an indexed for-loop and reduce. As I wrote, it's not just the absolute perf., it's the variability and unpredictability.
>Just summing floats in an array, compiled without optimization,
Ok, so you're both doing it wrong and doing something irrelevant to that level of systems programming. Measuring the performance of non-optimized builds will always be pretty meaningless (even C compilers can have radical perf swings from release to release at -O0), and reporting that as a rebuttal of whether swift is relevant for "bootloader or firmware" is just outright trying to mislead people.
> With optimization that went down to around 10x and wouldn't budge, even with -Ounchecked, UnsafeMutablePointer, varying an indexed for-loop and reduce.
Beyond what you said, I suspect that all of these impressions were formed with Swift 1. Believe it or not, things have improved a lot since then. On a "add array of floats" benchmark, the vectorizer kicking in or not is a 4x or more performance difference. This should be fixed in Swift 2, but more to the point is a benchmark irrelevant to "boot loaders and firmware".
> doing something irrelevant to that level of systems programming
Except: no. I reported the float-array result because it was literally the 2nd test I ran and hadn't gotten around to integer arrays yet. I was expecting this to be completely doozy, one of the things Swift should be acing by now. Of course I am aware that you wouldn't use floats in systems programming, but as you should know, floats today are simple machine-level scalars that present little if any overhead to the CPU compared to integers. That is, this test uses a few basic machine operations: indexed memory access, loops, scalar arithmetic.
Whether those scalars are floats or integers should be largely irrelevant to the outcome of the test and therefore the test is a good enough (if not perfect) proxy for the types of operations you would see in systems programming.
Are you seriously claiming that the results would be different if the scalars in question had been integers instead of floats? Seriously??
If you had made such a claim, you'd be wrong: I've now run the same test with integer arrays and the result is exactly the same. Next you are going to tell me that integers are irrelevant to systems programming?
> [non-optimized builds] and reporting that as a rebuttal of whether swift is relevant for "bootloader or firmware" is just outright trying to mislead people.
No, it is you who is misleading people by completely mischaracterizing what I wrote. I wrote that performance varies from 30% worse to 1000x worse "depending on optimization level". This is exactly what is happening. (Of course, for this precise test it is 10x worse to 1000x worse, but who's counting?)
Claiming that non-optimized builds are irrelevant is also disingenuous at best, because non-optimized is the default setting for non-release builds in Xcode. Having this type of performance (variance) means that debug builds are effectively unusable: an operation that would take 100ms in a release build would take close to two minutes in a debug build.
Yes, other languages also have different performance levels between debug and non-debug builds, but size matters. 2-10x is something you can deal with, 1000x is not.
Furthermore, the problem is not so much the 1000x worse performance (although that is funny), it is the difference between optimized and non-optimized builds. The optimizer is not part of the language spec, meaning you can't rely on it. Next compiler release the compiler can change a little bit so it won't optimize one part of the code that you needed to run fast. Or you change the code a little bit in a way that the optimizer doesn't like and the effect is the same. And of course the compiler doesn't tell you what optimizations ran or did not run, so you are dealing with the blackest of black boxes. One day your code is fast, the other day it is not.
This is not predictable performance, and today predictability is often more important than raw speed for performance work. See my Jitterdämmerung post[1] or The Death of Optimizing Compilers[2].
> Beyond what you said, I suspect that all of these impressions were formed with Swift 1
You suspect wrong, all this was using the newest Xcode tools (7.1.1 previously, just repeated with 7.2):
Apple Swift version 2.1.1 (swiftlang-700.1.101.15 clang-700.1.81)
Target: x86_64-apple-darwin15.2.0
And by suspecting wrong, you kinda make my point for me, so thank you, Chris! Swift's performance model is so amazingly unpredictable that even the language's creator, compiler- and optimizer-wizard extraordinaire can't figure it out. How are mere mortals supposed to do so?
When doing systems programming and embedded programming, predictability is paramount. I remember the story of the railway engineer who scoured the planet for remaining stocks of 8085 processors, because he thought he understood the bugs in that particular processor.
Now sure, you certainly can use Swift for these types of tasks despite all the issues. But I suspect you'd be better off using FORTH.
someone writing a bootloader or firmware can stick to using Swift structs and have a simple guarantee of no dynamic overhead or runtime dependence
There seems to be a pretty strong implication here that Swift could be used to write firmware/bootloaders, and other low level constructs - including operating systems. Has anyone worked with Swift yet on that type of project?