During the quest "Affairs of the Heart" in Hlerynhul, 

I gave the letter to Ulyne Rothalen after her in-law followed me inside,
I never mentioned Breynard Ancois to any of them,
When I go back to Breyard, the Journal said something like "Breynard was angry I didn't give the letter directly", but I did.

Is this because I was watched by her in-law ?
Should I have given her the letter with more discretion ? Should I have entered the house without her in-law noticing ?
The journal entry might be a bit clearer on that topic. Or maybe I wasn't suppose to get this entry after I gave the letter.

Here is the order of the entries I got

1. "I met a Breton knight..."
2. "I gave breynard's letter to Ulyne"
3. "Breynard was angry... asked me to meet him in two days"