Start by whisking honey, vanilla, orange juice, orange zest, and nutmeg in a medium saucepan. Heat over medium-high until it comes to a gentle boil, making sure the flavors meld together perfectly.
Once boiling, stir in the chopped walnuts and cook for about 4 minutes, stirring occasionally. This toasts the nuts lightly and infuses the syrup with a nutty crunch.
Place apricot halves on a serving plate and top each with a slice of goat cheese. Spoon a tablespoon of the warm syrup over each apricot for a glossy, irresistible finish.