I just recently moved into my first house this morning and when I went outside I noticed this:
What happened? What should I do? I have no idea what that pipe even is. I don't believe there is a septic tank on my property because we use city utilities?
My guess, and it's only a guess, is that that pipe is where the downspout used to attach to the sewer system. Until very recently, downspouts used to connect directly to the sewage system. Many municipalities have passed laws mandating the removal of these connections because the treatment plant would often overflow during a storm, leading to the dumping of untreated sewage into natural waterways.
So what's happened, is there is a block downstream of the connection between this storm sewer and the household sewer. Typically, if the blockage occurs on the road side of the water supply cut off -- 
-- then the city is responsible. If it's inside, then the homeowner is responsible.
In my city, they'll run a camera down the pipe, and charge you $300 for the scoping if the blockage is on your side. I'm betting in the short term, you'd be better of to get a city licensed plumbing and drainage contractor to snake it out and absorb the cost. Not a DIY job, because the city will make you pay for anything that goes wrong.
The next issue is how to stop it from happening again. You'll probably have to dig down to the connection, remove the downspout and cap it off. Again, since you're playing with municipal systems, this is not a DIY job.
You might get lucky with calling the city, but then again, they might inspect certain systems and order you to fix them right away, at a cost you likely can't afford if you just bought the house.
Whatever it is, it did it's job. Looks like the cap blew off (the white thing to the right) and the backed up sewage came out of the pipe instead of in your house! I am guessing they didn't use a screw on cap or glued on, for this reason. Maybe the sewer backing up happened often and the homeowner had this put in? Call a plumber to look into it more and fix the backup.
It could also be a cleanout for the sewer line from the house to the city main. The cleanout is used for easy access tho the main sewer line to clear blockages with a drain cleaning cable (snake).
