What does the young man mean by "great honey coloured/Ramparts at your ear?" Why does he say the young men are "thrown into despair" by them?

"By the honey-coloured ramparts", the young man means the golden coloured hair locks of his lover that hung around her ears. By “thrown into despair”, the poet means that the hairs were so attractive that young men gained a strong desire to love her. They loved her for her beautiful hair and not for what she actually was as a person.