I'm not sure why this tiny transaction cost so much more than the huge one, but being able to support 0.00045 BTC transactions at reasonable cost is the challenge.
This isn't just a fee challenge, but a blockchain-bloat challenge as well.
But if it can be solved, I think it would be a turning point for BTC.
It's not that transactions "incur" a fee. You pay the fee you want, and miners prioritize transactions accordingly. Because transactions take up space in the block chain (and everyone's hard drives), small transactions are given less priority. Large transactions usually don't need to pay fees to be mined, small transactions may take a long time to be mined (or in the case of "dust" not be mined at all) unless you pay the recommended fee.
I think because of the design of bitcoin $0.04 transactions are something they want to actively discourage. From what I understand of Bitcoin, the future will hold only large-scale transactions like this one, and as little sub-dollar transactions as possible. (because the network load of the system scales with the transaction quantity)
This, along with regulatory challenges, has hinted to me for at least a few years now that if bitcoin has any long term future at all, it will be as an exchange currency.
Its destiny is cryptogold where the store of value is in the security of sha256. In the future they will probably adopt new cryptography standards into the protocol as current encryption standards are broken down (ie, sha1 was used at first but you should not use it anymore due to its growing leakiness).
Is this an artifact of current network dynamics or is it explicit in the protocol?
In other words, can I, today, send zero-fee transactions and reasonably (99.99% of the time or something) expect them to be processed by some _known_ deadline? (Be it 24 hours or a week or whatever)
It is just current network dynamics. Confirmation times in general are unknown as blocks are produced on an arbitrary timescale that the software attempts to trend to a 10 minute average by actively changing the difficulty of the proof-of-work mechanism to match the entire hash power of the network.
For transactions without fees there is no way to guarantee anything.
There was once optimism that bitcoin would allow easy micropayments. This seems to be an issue with that, though I believe it's one that has been long recognized by those actually doing much with bitcoin.
For instance, there's this transaction for .02 BTC that incurred a $0.19 fee. https://blockchain.info/tx/a2933a1dc53361a7770c2a9d998c1fe30...
I'm not sure why this tiny transaction cost so much more than the huge one, but being able to support 0.00045 BTC transactions at reasonable cost is the challenge.
This isn't just a fee challenge, but a blockchain-bloat challenge as well.
But if it can be solved, I think it would be a turning point for BTC.