You could always use radio waves for the upload band. That way, it will still increase both download and upload speeds (uploads will can have a wider band now that they don't compete with downloads for the same airspace), it just won't increase upload speeds as much.
This is the right idea for sure. Data usage is almost always asymmetric - people download much more than they upload. So let's widen the download channel using visible light and still use the sufficiently large RF channel for the up stream.