Classic weathercocking, and you’ve got the cause right: short boats with lots of rocker get pushed around. Start with a skeg, not a rudder. Cheaper, simpler, no cables.
DIY: Easiest version: cut a skeg shape out of a $5 plastic cutting board, about 8 inches long and 3 inches deep. Bolt it through the keel line near the stern with stainless hardware and fender washers inside the hull, and seal the holes with marine sealant.
Purchase Option: tons of kayak rudders or skegs that flip up and down and mount in the stern. Don’t be afraid to drill into kayak. Just make sure to do so above the waterline. Some rudders and skegs come with a gudgeon mount. You mount that then the rudder or skeg drops in and a pin secures. This company makes gudgeon mounts