Thats due to the rotation of the earth, for the purposes of the simulation the earth is rotating at 408 m/s (earths rotation at cape canaveral) so thats why It appears its going sideways
When the rocket launches it already moves with earth's rotation at that latitude (and is stationary with respect to the ground).
That's in fact the reason why many rocket launch sites are near the equator: Free velocity.
But the rotational velocity of the rocket is going to be very similar to Earth's at the launch site. Eventually the rotational velocity between ground and the rocket will begin to noticeably differ, but the launch trajectory starting at 408 m/s would look terrifying if it were that way in real life.
I found the problem, It appears that I had the earth staying stationary but gave its rotational velocity to the rocket, What i think needs to happen is I need to also be rotating the earth so the inertial frame stays the same. Thanks for bringing this up