How to delete a wallpaper

In the Album view, where you can swipe between different wallpapers, long-press on the wallpaper you want to delete.

A context menu will popup with a Delete option.