Again about holey abstractions (or about an unpredictable environment)
So, a fairly simple part of the program for Windows. There is a file containing several entries. And they need to be filtered in a certain way.
The solution is quite simple - open the file, read the records one by one, we write the necessary ones to a temporary file. Close the file. We delete it. Rename the temporary to the original. Everything is so simple that I won’t even give the code. Is this really enough reason for the article?
While everything works, there is really no reason to write about it. But then suddenly one day “everything falls”, because renaming does not occur due to an Access denied error. This happens very rarely, but still much more often to suspect cosmic rays.
We begin the excavation. The first clue found: in the process of digging, this quote from the Microsoft documentation alerted:
The culprit was found thanks to SysInternals and their ProcessMonitor. We launch it, install the filter on our long-suffering file and try to reproduce it for a long and persistent. We reproduce. We look. And what do we see there?
On line 41 we close our file. In line 42 - delete. And since explorer.exe is still reading it, the file is not deleted. What we can see on lines 46 and 47 when we try to rename our temporary file to the main one (DELETE PENDING status instead of SUCCESS).
Explorer.exe finishes reading only on line 49. Only at that moment the file is physically deleted (as line 50 indirectly tells us, where our persistent explorer.exe again tries to open the file for reading, but it fails because the file not anymore).
What follows from this? Thanks to Microsoft Windows, even a simple file deletion operation now needs to be done in the style of paranoid programming. They called the delete function, made sure that it returned OK, and entered the wait cycle "until the file has been physically deleted yet." Well and yes, without knowing that “she has a neon inside,” it’s impossible to do practically anything ...
But now doubt is gnawing at me that such an approach is a common practice. In order to keep in mind during any work with files in Windows that the OS at any time decides to do something with the files without your knowledge, and write code that is resistant to this. In this connection, the survey is lower.
The solution is quite simple - open the file, read the records one by one, we write the necessary ones to a temporary file. Close the file. We delete it. Rename the temporary to the original. Everything is so simple that I won’t even give the code. Is this really enough reason for the article?
While everything works, there is really no reason to write about it. But then suddenly one day “everything falls”, because renaming does not occur due to an Access denied error. This happens very rarely, but still much more often to suspect cosmic rays.
We begin the excavation. The first clue found: in the process of digging, this quote from the Microsoft documentation alerted:
The DeleteFile function marks a file for deletion on close. Therefore, the file deletion does not occur until the last handle to the file is closed. Subsequent calls to CreateFile to open the file fail with ERROR_ACCESS_DENIED.In terms of symptoms, it’s very much alike, but where do these “other handlers” come from, if, apart from us, nobody does and should not do anything with this file? And we do not have any other threads with threads that would do something with this file?
The culprit was found thanks to SysInternals and their ProcessMonitor. We launch it, install the filter on our long-suffering file and try to reproduce it for a long and persistent. We reproduce. We look. And what do we see there?
01. 2: 25: 28.3162097 PM our_prog.exe 1288 CreateFile our.file SUCCESS Desired Access: Generic Read / WriteAnd we see the following there (extra data was deleted so as not to clutter up). Lines 1 to 36 - we create a file, write to it, make flush. The most interesting begins in lines 38-40. Explorer.exe appears from nowhere and starts reading our file.
02. 2: 25: 28.3164513 PM our_prog.exe 1288 WriteFile our.file SUCCESS Offset: 0, Length: 898, Priority : Normal
...
34. 2: 25: 28.3173405 PM our_prog.exe 1288 WriteFile our.file SUCCESS Offset: 36,403, Length: WriteFile our.file SUCCESS Offset: 35,290, Length: 1,113
35. 2: 25: 28.3173493 PM our_prog.exe 1,128
36. 2: 25: 28.3173736 PM our_prog.exe 1288 FlushBuffersFile our.file SUCCESS 37.2
: 25: 28.3174212 PM our_prog.exe 1288 WriteFile our.file SUCCESS Offset: 0, Length: 40,960,
38. 2: 25: 28.3175927 PM Explorer.EXE 1884 QueryBasicInformationFile our.file SUCCESS 39.2
: 25: 28.3176144 PM Explorer.EXE 1884 CloseFile our.file SUCCESS
40. 2: 25: 28.3263642 PM Explorer.EXE 1884 CreateFile our.file SUCCESS Desired Access: Read Attributes,
41. 2: 25: 28.3294990 PM our_prog.exe 1288 CloseFile our.file SUCCESS
42. 2: 25: 28.3351356 PM our_prog. exe 1288 CreateFile our.file SUCCESS Desired Access: Read Attributes, Delete,
43. 2: 25: 28.3351856 PM our_prog.exe 1288 QueryAttributeTagFile our.file SUCCESS Attributes: A, ReparseTag: 0x0
44. 2: 25: 28.3352020 PM our_prog.exe 1288 SetDispositionInformationFile our.file SUCCESS Delete: True
45. 2: 25: 28.3352218 PM our_prog.exe 1288 CloseFile our.file SUCCESS
46. 2: 25: 28.3358275 PM our_prog.exe 1288 CreateFile our.file DELETE PENDING Desired Access: Generic Read / Write
47.2: 25: 28.3362207 PM our_prog.exe 1288 CreateFile our.file DELETE PENDING Desired Access: Generic Read / Write,
48.2: 25: 28.3367696 PM Explorer.EXE 1884 QueryBasicInformationFile our.file SUCCESS
49. 2: 25: 28.4279152 PM Explorer.EXE 1884 CloseFile our.file SUCCESS
50. 2: 25: 28.4282859 PM Explorer.EXE 1884 CreateFile our.file NAME NOT FOUND Desired Access: Read Attributes,
...
83. 2: 25: 29.3497760 PM our_prog.exe 1288 CreateFile our .file SUCCESS Desired Access: Generic Read / Write,
On line 41 we close our file. In line 42 - delete. And since explorer.exe is still reading it, the file is not deleted. What we can see on lines 46 and 47 when we try to rename our temporary file to the main one (DELETE PENDING status instead of SUCCESS).
Explorer.exe finishes reading only on line 49. Only at that moment the file is physically deleted (as line 50 indirectly tells us, where our persistent explorer.exe again tries to open the file for reading, but it fails because the file not anymore).
What follows from this? Thanks to Microsoft Windows, even a simple file deletion operation now needs to be done in the style of paranoid programming. They called the delete function, made sure that it returned OK, and entered the wait cycle "until the file has been physically deleted yet." Well and yes, without knowing that “she has a neon inside,” it’s impossible to do practically anything ...
But now doubt is gnawing at me that such an approach is a common practice. In order to keep in mind during any work with files in Windows that the OS at any time decides to do something with the files without your knowledge, and write code that is resistant to this. In this connection, the survey is lower.
Only registered users can participate in the survey. Please come in.
Have you encountered a situation of deferred file deletion under Windows?
- 46.9% No 53
- 53% yes 60