Are semantics important

You need to think of it in terms of separating semantics (meaning) from presentation (design). You can create a terrible, messy HTML soup using only elements, but you can make that look really pretty. The presentation will be fine, but the semantics of the source code will be terrible.

Now, to whom do source code semantics matter? To anyone who's interacting with the source code but not the presentation. This can be search engines, which try to find the most relevant content of the page so other people can find you when searching for you. This can be screen readers which try to make content accessible to people who don't care about the visual presentation of your site but need to access the content in alternative ways. Try reading an HTML soup page with nothing but a Braille display and you'll soon learn a few new curse words.

It could also just be you, trying to efficiently maintain and expand your own code over time.

There isn't always one specific benefit or penalty, but you won't be able to reap any benefits at all unless you start producing semantic HTML. The future of the web is pages on many different kinds of devices. If you ever expect to perhaps be able to see relevant information from your website on your smart watch, that smart watch needs a good way to hide all extraneous stuff like navigation and headers and zoom in on the actual content quickly. There needs to be a standardised way to identify main content and auxiliary content in order to do that, and those HTML5 tags are that way.

If you know what something means you can present it in many different appropriate ways. If you just hardcode one look, it can only be presented in that one look.

Today the only advantage may be slightly more readable source code, tomorrow the advantage will be new use cases for your content that you couldn't even think about before. You're never going to get there though if you don't take semantics seriously.

Perhaps as a more tangible example: the Opera browser had (has?) an experimental feature in which it displays an additional bar below the address bar which contains buttons to navigate the page you're on; using it you could quickly jump to the top page, the next page, an index page etc. For that to work the browser obviously needs to know what the "top" or "next" page is. That's impossible to figure out without standards. See here for further explanation.