1640 is a lovely restaurant in the old town, just opposite to the Frontenac castle, which can offer a nice view if you seat next to the windows. I guess they might have a terrasse in the summer.
I went there twice for breakfast during my stay in Québec. The place is very cosy with a nice and sober decoration. The staff is super friendly and the breakfast menu quite appealling. I tried their brunch plate with scrambled eggs, bacon, ham, sausages, potatoes, toasts and a cupcake. Really really tasty. The 2nd time, I had almost the same thing but in smaller size.