A question the writer of narrative must always ask is this: What does the reader want? You ask the question because your job is to identify what the reader wants and refuse to give it over.
The reader does not want what she thinks she wants. The reader thinks she wants the two characters to kiss, but what the reader really wants is for the two characters to refrain from kissing while the possibility that they might still kiss remains present.
However, there is a limit to the reader's patience. If it becomes obvious that the writer is simply playing games, twisting things around so that the characters don't kiss, then the reader feels that the work is contrived and full of cheap tricks and too plot-driven and all of a sudden the desire to see the characters kiss dissipates. Instead of increasing, the tension dissolves as the reader grows bored.
For this reason, one of the greatest technical accomplishments of a narrative writer is to satisfy the reader while refusing absolutely to give the reader what she wants. To satisfy the reader with a deep ambiguity instead of the superficial certainties that the reader always insists upon but does not truly want. What the reader desires is something to do — something to imagine — a story structure that satisfies but which she can turn her mind to again and again and within which she can play with multiple possibilities.
The Turn of the Screw is exemplary in this regard. Although the actual prose and characterization are very of its time (bland and melodramatic for the most part), the plot is structured brilliantly.
The book begins, as many horror stories do, with a frame narrative, where one character testifies to the strangeness of the following story, although oddly in this case does not make grand claims for its veracity or otherwise stand as a witness. The frame story never returns, as a result of James “getting out” of the novel at its absolute climax — a masterful show of restraint, as James refuses, in returning to the frame narrative or even playing out the aftermath of the story's climax, to settle the questions the climax raises.
The Turn of the Screw is, in essence, a ghost story, although there is some question as to whether or not the ghosts themselves are “real” or fantasies. There is also the question, as in all ghost stories, of what the ghosts want — what are their powers, what are their plans? James manages to build, through a series of unsettling scenes (one in particular is quite troubling, though its materials are commonplace), a great desire to know the details of the haunting.
And yet James never delivers. Instead, the story plays out according to its established logic — but without being too predictable — and culminates in a climax that raises more questions than it answers, and yet serves as a satisfying conclusion. Just where another writer would have begun a new “act,” moving on to “wrap up” all the loose ends, James lets the threads dangle and ends the book.
It works because the event more or less severs the plot — there is nothing else the characters can meaningfully do, aside from answer questions. For better or worse, the ghosts have succeeded (if, indeed, there are ghosts… James leaves us room to question if this is all the result of some madness), and so all the characters could do is work to figure out what the ghosts had to gain, why they carried out this plan, and make sense of things in retrospect — but there's nothing the characters could do with the knowledge, so James lets the loose ends dangle. And we pick them up. As readers we look back over the story, try to puzzle it out.
As in a David Lynch film, or in psychosis, there appears to be an internal logic to the story's events, and yet it's not obvious to an outside viewer/reader. So we have to try to put ourselves into the madness. This is where a horror story can be particularly effective. Such stories are always more disturbing when there appears to be some reason for the unfolding of events that appears “logical” but cannot be understood or grasped.
Think of Hitchcock's film “The Birds.” Why do the birds attack? The attacks don't seem random — there appears to be a logic to them, but though we obsessively try to untangle that logic it will always elude us. We never receive a satisfactory answer, and if we did, the birds would immediately be less interesting — and less frightening.
The Turn of the Screw isn't quite the masterpiece I might seem to suggest here — it's often clunky, a lot of the scenes meant to be disturbing are simply boring, and the characters for the most part are flat — but in terms of its structural ambiguity it's an excellent example.