It depends on the player. Just try it and see for yourself.

As just for data dvds, as sleepless said, depends on player, my dvd player can play divx and mpeg 4 files but not files labelled avi. It plays them in a similar fashion to mp3 dvd's, with a similar menu.
The big advantage of course is you can fit so much more on. eg: Music Videos around 600 MB, I could fit around 7 on a dvd. But if burning a normal dvd video format (mpeg2), it would hold a fraction of that. The disadvantage is you need to re-access the menu for each thing you wish to play which is a nuisance with short video clips such as individual music video songs. Program a list of desired is only way around that continual interruption.