In my opinion a bad book is worst than a did not finish book. With a bad book there is a possibility that your book-discussing-friend might be interested in this "horrendous" book. It sounds so bad that the mind thinks, 'how bad can it be?'.
Whereas with a did-not-finish book what can you say to your friend when they ask about it? It was eh...blah...I didn't get anything out of it. There is nothing you can say to intrigue your friend. It all comes down to 'curiosity killed the cat'.
So in your eyes, which is worst? A book that you were not able to finish reading or a book that you considered bad, but was still able to finish it. Or are these two categories correlated?