jos/lab5.html at lab5 · wky/jos · GitHub

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
59
60
61
62
63
64
65
66
67
68
69
70
71
72
73
74
75
76
77
78
79
80
81
82
83
84
85
86
87
88
89
90
91
92
93
94
95
96
97
98
99
100
101
102
103
104
105
106
107
108
109
110
111
112
113
114
115
116
117
118
119
120
121
122
123
124
125
126
127
128
129
130
131
132
133
134
135
136
137
138
139
140
141
142
143
144
145
146
147
148
149
150
151
152
153
154
155
156
157
158
159
160
161
162
163
164
165
166
167
168
169
170
171
172
173
174
175
176
177
178
179
180
181
182
183
184
185
186
187
188
189
190
191
192
193
194
195
196
197
198
199
200
201
202
203
204
205
206
207
208
209
210
211
212
213
214
215
216
217
218
219
220
221
222
223
224
225
226
227
228
229
230
231
232
233
234
235
236
237
238
239
240
241
242
243
244
245
246
247
248
249
250
251
252
253
254
255
256
257
258
259
260
261
262
263
264
265
266
267
268
269
270
271
272
273
274
275
276
277
278
279
280
281
282
283
284
285
286
287
288
289
290
291
292
293
294
295
296
<!doctype html>
<html>
<head>
<meta charset="utf-8">
<meta name="viewport" content="width=device-width, initial-scale=1.0, user-scalable=yes">
<style>
h1,
h2,
h3,
h4,
h5,
h6,
p,
blockquote {
    margin: 0;
    padding: 0;
}
body {
    font-family: "Helvetica Neue", Helvetica, "Hiragino Sans GB", Arial, sans-serif;
    font-size: 13px;
    line-height: 18px;
    color: #737373;
    background-color: white;
    margin: 10px 13px 10px 13px;
}
table {
	margin: 10px 0 15px 0;
	border-collapse: collapse;
}
td,th {
	border: 1px solid #ddd;
	padding: 3px 10px;
}
th {
	padding: 5px 10px;
}

a {
    color: #0069d6;
}
a:hover {
    color: #0050a3;
    text-decoration: none;
}
a img {
    border: none;
}
p {
    margin-bottom: 9px;
}
h1,
h2,
h3,
h4,
h5,
h6 {
    color: #404040;
    line-height: 36px;
}
h1 {
    margin-bottom: 18px;
    font-size: 30px;
}
h2 {
    font-size: 24px;
}
h3 {
    font-size: 18px;
}
h4 {
    font-size: 16px;
}
h5 {
    font-size: 14px;
}
h6 {
    font-size: 13px;
}
hr {
    margin: 0 0 19px;
    border: 0;
    border-bottom: 1px solid #ccc;
}
blockquote {
    padding: 13px 13px 21px 15px;
    margin-bottom: 18px;
    font-family:georgia,serif;
    font-style: italic;
}
blockquote:before {
    content:"\201C";
    font-size:40px;
    margin-left:-10px;
    font-family:georgia,serif;
    color:#eee;
}
blockquote p {
    font-size: 14px;
    font-weight: 300;
    line-height: 18px;
    margin-bottom: 0;
    font-style: italic;
}
code, pre {
    font-family: Monaco, Andale Mono, Courier New, monospace;
}
code {
    /*background-color: #fee9cc;*/
    color: rgba(0, 0, 0, 0.75);
    padding: 1px 3px;
    font-size: 12px;
    -webkit-border-radius: 3px;
    -moz-border-radius: 3px;
    border-radius: 3px;
}
pre {
    display: block;
    padding: 14px;
    margin: 0 0 18px;
    line-height: 16px;
    font-size: 11px;
    border: 1px solid #d9d9d9;
    white-space: pre-wrap;
    word-wrap: break-word;
}
pre code {
    background-color: #fff;
    color:#737373;
    font-size: 11px;
    padding: 0;
}
sup {
    font-size: 0.83em;
    vertical-align: super;
    line-height: 0;
}
* {
	-webkit-print-color-adjust: exact;
}
@media screen and (min-width: 914px) {
    body {
        width: 854px;
        margin:10px auto;
    }
}
@media print {
	body,code,pre code,h1,h2,h3,h4,h5,h6 {
		color: black;
	}
	table, pre {
		page-break-inside: avoid;
	}
}
</style>
<title>MIT 6.828 JOS Lab5</title>

</head>
<body>
<h1>MIT 6.828 JOS Lab5</h1>

<p><strong>YANG Weikun 1100012442</strong></p>

<p>14th Oct 2013 02:00 UTC+0800</p>

<h2>Getting Started</h2>

<p>After merging with <code>origin/lab5</code>, <code>GNUmakefile</code> uses native gcc to compile something for the disk image. The gcc on my system is <code>x86_64-apple-darwin13.0.0-gcc</code>.</p>

<h2>Ex.1 Disk Access</h2>

<p>Following the monolithic kernel design, we are moving the file system out side the OS. Disk access are all handled in a special user process with type <code>ENV_TYPE_FS</code>.  We use polling I/O to further reduce the complexity.</p>

<p><code>IOPL</code> bits in <code>EFLAGS</code> register control device I/O privilege. So the FS process will be the only process initiated with such privilege.</p>

<h3>Q&amp;A</h3>

<ol>
<li><em>'Do you have to do anything else to ensure that this I/O privilege setting is saved and restored properly when you subsequently switch from one environment to another? Why?'</em>

<ul>
<li>Nope, I didn't need to do anything else. When an interrupt occurs the <code>EFLAGS</code> register is saved automatically by hardware. When the kernel returns to user programs <code>EFLAGS</code> is restored.</li>
</ul>
</li>
</ol>


<hr />

<h2>Ex.2 The Block Cache</h2>

<p>Our block cache strategy can't be simpler: we use 3GB of memory space from <code>DISKMAP</code>, to map the entire disk. Access to disk block <code>i</code> will try to access memory location <code>DISKMAP + i * BLKSIZE</code>. If the block does not exist in memory a page fault will occur, then our page fault handler will allocate space and read the actual disk contents in memory.</p>

<p>In this exercise I encountered several strange problems, one of which is that <code>ROUNDDOWN</code> causes illegal memory access when code is compiled with <code>-O0</code>. Never figured out why, so I didn't use <code>ROUNDDOWN</code> anymore.</p>

<hr />

<h2>Challenge: File system with write access</h2>

<h3>JOS filesystem overview</h3>

<ol>
<li><strong>File Operations</strong>(for user programs) <code>open</code>, <code>close</code>, <code>seek</code>, <code>read</code>, <code>write</code>, <code>stat</code> and other functions are visible to user programs, to accomplish file operations using <code>fd</code>(file descriptors). Most of those functions are delegates to device specific handlers. They are defined in <code>fd.c</code> and <code>file.c</code>. There are many other functions to manage a process's <code>fd</code>s.</li>
<li><strong>Device Abstraction</strong> Files, Pipes, Consoles are the three devices in JOS. Each type of device has its own handlers for various operations. <code>devfile_read</code>, <code>devfile_write</code> etc. are handlers of File device (in <code>file.c</code>). Those functions invoke IPC to the file server.</li>
<li><strong>IPC to File Server</strong> Upon receiving a request from a user process, the file server will interpret the requested job, dispatch the actually work to the next level. The file server is in <code>serv.c</code>.</li>
<li><strong>File Operations</strong> This level will walk through directories to open a file, locate blocks for read/write. This part is in <code>fs.c</code></li>
<li><strong>Block Cache</strong> The entire disk is cached in the file server: <code>[DISKMAP, DISKMAP+DISKSIZE)</code>. The file server will try the memory for read/write of blocks, the block cache will load blocks into memory when necessary. Write-back and eviction is added to enable a writable filesystem.</li>
<li><strong>On Disk Structure</strong> On the disk, block 0 is the MBR. Block 1 is the superblock, containing the size of the disk and the root directory. starting from block 2 lies the bitmap, to keep track of allocated blocks and free blocks. Then comes blocks used for meta-data and contents of files.</li>
<li><strong>IDE Driver</strong> The driver issues IDE commands to read/write physical blocks.</li>
</ol>


<h3>Changes made to support writable filesystem.</h3>

<ol>
<li><strong>Device level operations</strong> handlers for write and truncate are added (<code>devfile_write</code> and <code>devfile_trunc</code>), both are simple wrappers of corresponding filesystem IPC calls.</li>
<li><strong>File Server</strong> the file server now may <code>open</code> a file with <code>O_CREAT</code>, <code>O_TRUNC</code>, <code>O_WRONLY</code> and <code>O_RDWR</code> options. All FS requests are handled appropriately.</li>
<li><strong>File Operations</strong> in <code>fs.c</code>:

<ul>
<li><code>alloc_blk</code>: find a free block in the bitmap</li>
<li><code>free_blk</code>: free a disk block</li>
<li><code>block_write_back</code>: write back changes to disk (if any), and un-map the corresponding memory page to release memory(the superblock is never unmapped).</li>
<li><code>file_block_walk</code>: find the pointer to the disk block that holds contents of a block in file. may allocate an indirect block if asked.</li>
<li><code>file_get_block</code>: find the actually disk block of a file, allocate one if none exists.</li>
<li><code>file_create</code>: create a new file or directory. allocate a new directory entry and create a file.</li>
<li><code>file_write</code>: write contents of buffer to appropriate location in file. File size may grow if writing past EOF.</li>
<li><code>file_set_size</code>: truncate (delete content blocks) or extend (allocate content blocks) the size of a file.</li>
<li><code>file_flush</code>: commit all changes on a file to disk.</li>
<li><code>fs_sync</code>: commit all changes on the filesystem to disk.</li>
</ul>
</li>
<li><strong>Block Cache</strong> when the block cache is out-of-memory, the file server will call <code>fs_sync</code> to free memory.</li>
</ol>


<h3>Results</h3>

<p>The filesystem in JOS is now able to create new files and directories, write data to files, truncate or extend a file. The simplest test would be:</p>

<pre><code>$cat &gt; newfile
blah blah blah
...

$ls
...
newfile
...

$cat newfile
blah blah blah
...
</code></pre>

<h2>Ex.3 Spawning Processes</h2>

<p>To implement a user mode <code>spawn</code> (equivalent to UNIX <code>fork + exec</code>), we must provide an interface for user programs to modify the trap frame of another user program, the system call <code>sys_env_set_trapframe</code>. In this system call, we first validate the pointer from user, then make sure that the modified trap frame has:</p>

<ol>
<li><code>FL_IF</code> on to receive interrupts.</li>
<li><code>FL_IOPL_MASK</code> off (to deny I/O operations).</li>
<li><code>CPL</code> of 3 so the program will run in user mode.</li>
</ol>


<hr />

<h2>Ex.4 Sharing library state across fork and spawn</h2>

<p>Following instruction from MIT, we must find pages marked with <code>PTE_SHARE</code> and map them accordingly in both <code>fork</code> and <code>spawn</code>.</p>

<hr />

<h2>Ex.5 The keyboard interface</h2>

<p>Input from QEMU window will be sent as keyboard interrupts with number <code>IRQ_OFFSET+IRQ_KBD</code>, handled by <code>kbd_intr</code>; Input from terminal window (of the host) will be sent as serial interrupts with number <code>IRQ_OFFSET+IRQ_SERIAL</code>, handled by <code>serial_intr</code>. <code>trap_dispatch</code> is modified to suit the change.</p>

<h3>Q&amp;A</h3>

<ol>
<li><em>'How long approximately did it take you to do this lab?'</em>

<ul>
<li>A bit more than 3 hours (without doing the challenges). This seems the easiest lab in all 5.</li>
<li>Writable filesystem challenge took some time. I had to understand the whole filesystem implementation, then make appropriate changes.</li>
</ul>
</li>
<li><em>'Any suggestions?'</em>

<ul>
<li>All 5 labs are great! Wonder if 64-bit <code>jos</code> is coming next year?</li>
</ul>
</li>
</ol>

</body>
</html>