Hardware Makes Software Hard
First off without hardware there can be no software. I also have great respect for all of the EEs out there and MS types who figure out how to bend physical reality to our will. You guys rock hard (bad pun intended). In terms of engineering prowess and practice most hardware manufacturing puts software development to shame.
That said, most of what is wrong in computer programming is your fault.
Ok, granted it wouldn't be possible to write or read this without your hard work, but the software used to deliver the experience is fundamentally an emergent property of that hardware. Most programmers don't understand how the system you designed works. They don't even know what an instruction set is let alone know the archaic one your processor translates into microcode. Hell your gate design is so far from what they understand, they don't even grasp the concept of timing. You know that instruction pipeline you keep having to fill, reordering microcode instructions to drive the right traces? Yeah, they typically don't grasp what happens to performance when they write code that stalls it. Nor are they capable of doing the timing math to figure it out.
So why is there this chasm?
Software programmers since P-code in Pascal have longed for a machine they have both control over and understand. Some programmers like Chuck Moore have gone so far as to design their own. When programmers build these systems they tend to construct shallow pipeline, random access, stack oriented, wire driven designs. Why? Because they don't need a bazillion clock cycles if they aren't burnning 32 machine instructions per VM instruction. To put that in perspective, a machine that can execute one VM instruction per clock cycle at 100MHz is equivalent to a VM running on a typical modern machine running at 3.2GHz. Yes software programmers are choosing to live in the Pentium days in order to control their platform.
What our hardware has become phenomenally good at is special purpose processing. Video/audio, cryptography, 3d graphics, and anything else software programmers typically use with big flashy presentations that speak well to the marketing department.
Where are the CPU instructions for vtable dispatch? Where are the special prefetch cache lines for JIT targets? Why do programmers write polymorphic inline caches, can't hardware tables map those? Where is my page table per object? Why can't each object run in it's own address space and have it's own CPU thread? Where is my 10,000 core Smalltalk machine? I mean we have enough transistors.
The problems with software stem from not understanding the hardware, and a cultural bias that values flashy over useful. Much of our best software engineering ideas are still in their infancy because the hardware is wasteful, geared towards marketing campaigns and a vicious cycle of benchmarks and gamesmanship. When was the last time you saw a benchmark in the trade press: "method dispatches per second"? You'd be hard pressed to find it outside of an ancient JVM chip marketing piece. When was the last time you saw a chip billed on context switches per second?