Just a few things to add to ddabrahim’s detailed answer…
It’s the easiest way, but I suppose you could move the background past the player by changing its X position, rather than changing the player’s X position. You could also have a repeating background (like they used to do in old cartoons) where you have a short section of background that is moved back to the start once you reach the end of it.
See: Infinite moving background - #2 by Lizard-13
Since you can have multiple layers and each layer has its own camera, different bits of background can move at different speeds (see image below).
Here one camera follows the player, by being constantly repositioned to the Player.X() position, while the “background” layer camera lags behind a bit, giving a parallax effect. Much more comprehensive coverage of this is in the wiki.
By “background” do you mean a single very large sprite, or lots of small objects placed next to each other to build a background?