To resume my attempt to erase the distinction between utterance and context, I shall argue that to judge whether a sentence is syntactically correct can require information about what looks like "context".
First, note that the fact that a given uttered sequence of sounds is in language or dialect L1 rather than in some other language L2 is surely very much a contextual feature, and one that has to be known in order to judge syntactic correctness. It may be that the speaker announced which language she was speaking, or that she is assumed to be speaking the same language as her interlocutor previously was speaking, or that she is speaking the majority language in the culture. Often one can guess from the words said, or the accent with which they are said, what language is being spoken. Often, but not always. Consider a case where we have two closely cognate languages. Then, one will sometimes have a case that the same set of sounds could be parsed either as a correct sentence of one language or as a somewhat mispronounced sentence of the other language. Which is the right interpretation depends on which language was contextually established as the one in which this utterance is being, and this in turn will answer the question whether the utterance was syntactically correct. So, if we recognize such a thing as context at all, we should likewise recognize as part of the context the fact that a givien language was in play, and hence we should conclude that context is relevant to syntax.
Second, we can come up with some somewhat odd-ball cases. You say: "Jones was walking" and I add "under the bridge". Whether what I said was syntactically correct depends on what you had said--had you said nothing, my addition would have been nonsense. So, what you say helps determine syntactic correctness.
Third, it seems to me that in gendered languages to use the wrong gender in words that refer to the speaker is to make a syntactic mistake--but then whether a sentence is syntactically correct will depend on whether the speaker is male or female, an apparently contextual feature. We can imagine even more radical versions of this--we can imagine a language where, say, completely different word order is to be used by men and by women. Similarly, it seems a syntactic mistake for a collective to speak in the first person singular. However, I am aware that there are other ways of interpreting the gender/number case (one might say that all the sentences in question are syntactically correct regardless of who says them, but there is some other kind of error in them).
So what? Well, if we need to know what we normally think of as context to determine syntactic correctness, then it seems that the choice of the setting in which we utter a set of noises is just as much a linguistic choice as the choice of what noises to make, because the setting and the noises interact to produce a syntactically correct or incorrect sentence.