xv6

port of xv6 to x86-64
git clone http://frotz.net/git/xv6.git
Log | Files | Refs | README | LICENSE

TRICKS (3925B)


      1 This file lists subtle things that might not be commented 
      2 as well as they should be in the source code and that
      3 might be worth pointing out in a longer explanation or in class.
      4 
      5 ---
      6 
      7 [2009/07/12: No longer relevant; forkret1 changed
      8 and this is now cleaner.]
      9 
     10 forkret1 in trapasm.S is called with a tf argument.
     11 In order to use it, forkret1 copies the tf pointer into
     12 %esp and then jumps to trapret, which pops the 
     13 register state out of the trap frame.  If an interrupt
     14 came in between the mov tf, %esp and the iret that
     15 goes back out to user space, the interrupt stack frame
     16 would end up scribbling over the tf and whatever memory
     17 lay under it.
     18 
     19 Why is this safe?  Because forkret1 is only called
     20 the first time a process returns to user space, and
     21 at that point, cp->tf is set to point to a trap frame
     22 constructed at the top of cp's kernel stack.  So tf 
     23 *is* a valid %esp that can hold interrupt state.
     24 
     25 If other tf's were used in forkret1, we could add
     26 a cli before the mov tf, %esp.
     27 
     28 ---
     29 
     30 In pushcli, must cli() no matter what.  It is not safe to do
     31 
     32   if(cpus[cpu()].ncli == 0)
     33     cli();
     34   cpus[cpu()].ncli++;
     35 
     36 because if interrupts are off then we might call cpu(), get
     37 rescheduled to a different cpu, look at cpus[oldcpu].ncli,
     38 and wrongly decide not to disable interrupts on the new cpu.
     39 
     40 Instead do 
     41 
     42   cli();
     43   cpus[cpu()].ncli++;
     44 
     45 always.
     46 
     47 ---
     48 
     49 There is a (harmless) race in pushcli, which does
     50 
     51 	eflags = readeflags();
     52 	cli();
     53 	if(c->ncli++ == 0)
     54 		c->intena = eflags & FL_IF;
     55 
     56 Consider a bottom-level pushcli.  
     57 If interrupts are disabled already, then the right thing
     58 happens: read_eflags finds that FL_IF is not set,
     59 and intena = 0.  If interrupts are enabled, then
     60 it is less clear that the right thing happens:
     61 the readeflags can execute, then the process
     62 can get preempted and rescheduled on another cpu,
     63 and then once it starts running, perhaps with 
     64 interrupts disabled (can happen since the scheduler
     65 only enables interrupts once per scheduling loop,
     66 not every time it schedules a process), it will 
     67 incorrectly record that interrupts *were* enabled.
     68 This doesn't matter, because if it was safe to be
     69 running with interrupts enabled before the context
     70 switch, it is still safe (and arguably more correct)
     71 to run with them enabled after the context switch too.
     72 
     73 In fact it would be safe if scheduler always set
     74 	c->intena = 1;
     75 before calling swtch, and perhaps it should.
     76 
     77 ---
     78 
     79 The x86's processor-ordering memory model 
     80 matches spin locks well, so no explicit memory
     81 synchronization instructions are required in
     82 acquire and release.  
     83 
     84 Consider two sequences of code on different CPUs:
     85 
     86 CPU0
     87 A;
     88 release(lk);
     89 
     90 and
     91 
     92 CPU1
     93 acquire(lk);
     94 B;
     95 
     96 We want to make sure that:
     97   - all reads in B see the effects of writes in A.
     98   - all reads in A do *not* see the effects of writes in B.
     99  
    100 The x86 guarantees that writes in A will go out
    101 to memory before the write of lk->locked = 0 in 
    102 release(lk).  It further guarantees that CPU1 
    103 will observe CPU0's write of lk->locked = 0 only
    104 after observing the earlier writes by CPU0.
    105 So any reads in B are guaranteed to observe the
    106 effects of writes in A.
    107 
    108 According to the Intel manual behavior spec, the
    109 second condition requires a serialization instruction
    110 in release, to avoid reads in A happening after giving
    111 up lk.  No Intel SMP processor in existence actually
    112 moves reads down after writes, but the language in
    113 the spec allows it.  There is no telling whether future
    114 processors will need it.
    115 
    116 ---
    117 
    118 The code in fork needs to read np->pid before
    119 setting np->state to RUNNABLE.  
    120 
    121 	int
    122 	fork(void)
    123 	{
    124 	  ...
    125 	  pid = np->pid;
    126 	  np->state = RUNNABLE;
    127 	  return pid;
    128 	}
    129 
    130 After setting np->state to RUNNABLE, some other CPU
    131 might run the process, it might exit, and then it might
    132 get reused for a different process (with a new pid), all
    133 before the return statement.  So it's not safe to just do
    134 "return np->pid;".
    135 
    136 This works because proc.h marks the pid as volatile.