learning_tla/lamport_video/docs/lecture10-2.md

# 10.2 Implementation with refinement - Refinement mappings

The AB2 protocol implements the ABSpec spec. But it also implements the AB protocol,
where a `LoseMsg` step of AB is implemented by a `CorruptMsg` step of AB2.

The spec says nothing about who performs which step. We naturally tend to think
that :

- sender A performs ASnd and ARcv steps
- sender B performs BSnd and BRcv steps
- the communications infra performs the LoseMsg step

But that is just an interpretation, suggested by the way the next step
is a disjunction of subactions.

How can we check that the AB2 protocol implements the AB protocol,
where `CorruptMsg` steps implement `LoseMsg` steps ? For this we need to
answer the following questions:

- what does this mean ?
- how can we check it ?

## Meaning

For every behavior of AB2 we can obtain a behavior of AB by replacing the state as follows:

| AB2 state                           | AB state                |
|-------------------------------------|-------------------------|
| `AVar = << "Tom", 1 >>`             | `AVar = << "Tom", 1 >>` |
| `BVar = << "Ann", 0 >>`             | `BVar = << "Ann", 0 >>` |
| `AtoB2 = << Bad, << "Tom", 1 >> >>` | `AtoB = << "Tom", 1 >>` |
| `BtoA2 = << 0, Bad, 0, Bad >>`      | `BtoA = << 0, 0 >>`     |

Saying that AB2 implements AB means that this transformation of AB2 states to AB states
transforms any behavior of AB2 into a behavior of AB.

To check that this is the case, we first transform states of the AB2 protocol
to produce behaviors satisfying a new specification `SpecH`.

| AB2 state                           | SpecH state                         |
|-------------------------------------|-------------------------------------|
| `AVar = << "Tom", 1 >>`             | `AVar = << "Tom", 1 >>`             |
| `BVar = << "Ann", 0 >>`             | `BVar = << "Ann", 0 >>`             |
| `AtoB2 = << Bad, << "Tom", 1 >> >>` | `AtoB2 = << Bad, << "Tom", 1 >> >>` |
| `BtoA2 = << 0, Bad, 0, Bad >>`      | `BtoA2 = << 0, Bad, 0, Bad >>`      | 
| <none>                              | `AtoB = << "Tom", 1 >>`             |
| <none>                              | `BtoA = << 0, 0 >>`                 |

The AB2 protocol implements the AB protocol iff every behavior allowed by `SpecH` 
is a behavior of the AB protocol: 
```tla+ code fragment
ABS == INSTANCE ABSpec
THEOREM SpecH => ABS!Spec
```

## Checking
### Preparing

We need to write SpecH and check it. A behavior should satisfy SpecH iff:

- The values of AVar, BVar, AtoB2, BtoA2 satisfy the AB2 spec: that is just AB2 `Spec`
- In every state
  - AtoB = AtoB2 without the corrupted messages
  - BtoA = BtoA2 without the corrupted messages
  - written: `[]((AtoB = RemoveBad(AtoB2)) /\ (BtoA = RemoveBad(BtoA2)))`

To permit A2BH to import `Spec` from AB2, it needs the same modules, constants,
and variables as AB2: Integers, Sequences, Data, Bad, the Bad assumption, AVar, BVar, and BtoA2,
after which it can import it using `AS2 == INSTANCE AB2`.

This makes the AB2 spec available to A2BH as `AB2!Spec`.

To the import AB, it also needs the missing variables `AtoB` and `BtoA`.

To apply the transformations we need as seen above, we need to define `RemoveBad
to transform states.

This allows us to define SpecH:

```tla+ code fragment
SpecH == /\ AB2!Spec
         /\ [] /\ AtoB = RemoveBad(AtoB2)
               /\ BtoA = RemoveBad(BtoA2)        
```

Here `AtoB` and `BtoA` are imaginary variables, added to `AB2!Spec` to show
that it implements the AB protocol spec. They are NOT meant to be implemented 
by the AB2 protocol.

If we ignore their values, then `SpecH` and `AB2!Spec` allow the same behaviors.

Note that this is NOT how AB2H writes it, although the result is the same:
```tla+ code fragment
SpecH == /\ AB2!Spec
         /\ [] /\ AtoB  \in Seq(Data \X {0,1})
               /\ BtoA \in Seq({0,1})
```

### Checking implementation: not checkable

Our goal is to check: `THEOREM SpecH => AB!Spec`, assuming we imported module AB
using `AB == INSTANCE AB`.

TLC CANNOT check this theorem, because `SpecH` does not have the standard form
for a TLA+ safety spec, which would be `InitH /\ [][NextH]_varsH`, with an 
initial state formula and a next state action.

To check the theorem, we need to rewrite it in a form that TLC can check: 
that is SpecHH.

### Simplifying refinement

Currently, we have:

```tla+ code fragment
SpecH == /\ AB2!Spec
         /\ [] /\ AtoB = RemoveBad(AtoB2)
               /\ BtoA = RemoveBad(BtoA2)        
AB == INSTANCE AB
THEOREM SpecH => AB!Spec \* not checkable
```
- 
- Since SpecH implies the _always_ formula (AtoB = ... BtoA2), it also implies
  `AB!Spec = (AB!Spec WITH AtoB <- RemoveBad(AtoB2), BtoA <- RemoveBad(BtoA2))`
- which means we can rewrite the theorem as:
  `THEOREM SpecH => (AB!Spec WITH AtoB <- RemoveBad(AtoB2), BtoA <- RemoveBad(BtoA2))`
- Since the formula no longer contains `AtoB` or `BtoA`, the whole always conjunct
  is irrelevant, and we can simplify the theorem to: `SpecH == AB2!Spec`
- which mean we can rewrite the theorem as:
  `THEOREM AB2!Spec => (AB!Spec WITH AtoB <- RemoveBad(AtoB2), BtoA <- RemoveBad(BtoA2))`
  - SpecH doesn't appear in the theorem anymore
- Let's move the statements to module AB2: we can remove the `AB!` namespace:
  - `THEOREM Spec => (Spec WITH AtoB <- RemoveBad(AtoB2), BtoA <- RemoveBad(BtoA2))`
- Now, in AB2, `Spec` is an ordinary safety spec, and TLC can check it
- However, this ` WITH AtoB <- RemoveBad(AtoB2), BtoA <- RemoveBad(BtoA2)` is not legal TLA+
- However, in AB2, we can do it at import time:
  - `AB == INSTANCE AB WITH AtoB <- RemoveBad(AtoB2), BtoA <- RemoveBad(BtoA2)`
- giving: `THEOREM Spec => AB!Spec` , which TLC can check

### Summary

- The AB2 protocol implements the AB protocol, where
  - `RemoveBad(AtoB2)` implements `AtoB` and
  - `RemoveBad(BtoA2)` implements `BtoA`
- This means that this theorem is true, the formula in the middle being what we called `SpecH`:
```tla+ code fragment
THEOREM (
    /\ AB2!Spec
    /\ [] /\ AtoB = RemoveBad(AtoB2)
          /\ BtoA = RemoveBad(BtoA2)
) => AB!Spec 
```
- Within module AB2, this is equivalent to:
```tla+ code fragment
AB == INSTANCE AB WITH AtoB <- RemoveBad(AtoB2), BtoA <- RemoveBad(BtoA2) 
THEOREM Spec => AB!Spec
```
- These `WITH` during the import are called a _refinement mapping_
  - We say "the AB2 protocol implements the AB2 protocol under this refinement mapping".
- TLC can check a model with `Spec` as the model `SPECIFICATION`, and `AB!Spec` 
  as the temporal property to be checked.

In general, if a spec `Spec2` does not contain all the variables in a spec `Spec1`,
then `Spec2` can only implement `Spec1` under a refinement mapping that assigns
expressions of `Spec2` to all the variables in `Spec1` that are not also in `Spec2`.
This is the usual case.

Even if `Spec1` and `Spec2` have a variable `v` in common, the refinement mapping
might substitute an expression of `Spec2` other than for the variable `v` in `Spec1`.

## Meaning, redux

What does it mean for an actual program to implement a TLA+ specification `Spec` ?

It means that we can, in principle, write a TLA+ specification `SpecPgm` of tha program,
and have `SpegPgm` implement `Spec` under a suitable refinement mapping.

We cannot do that in practice, because such a spec would be much too long and
complicated, but understanding refinement mappins can help prevent implementation errors.
Even if we cannot write the refinement mapping in TLA+, we should be able to explain
INFORMALLY how the spec's variables are implemented by the program's state.

Such an informal refinement mapping explains what the program is doing.
Writing it down, as comments in the code, CAN expose errors in the program.

## Imaginary variables

In A2BP we added imaginary variables `AtoBgood` and `BtoAgood` to the AB2 protocol spec,
to obtain `SpecP`, so that we could write a liveness property.

Similarly, we added imaginary variables `AtoB` and `BtoA` to the AB2 protocol spec,
to obtain `SpecH`, to show that the AB2 protocol safety spec implements the AB protocol
safety spec.

More generally, the fact that `Spec2` can be obtained by adding imaginary variables
to `Spec1` means that `Spec2` and `Spec1` allow the same behaviors if we ignore
the values of the imaginary variables.

We did that to show that the AB2 spec implements the AB spec, but that was not
necessary, because we could have used a refinement mapping instead. 

But sometimes, we have to add imaginary variables to define a refinement mapping.
Ignoring liveness, the AB and AB2 protocols are essentially the same, so `Spec` 
of AB should implement `Spec` of AB2 under some refinement mapping.

We shwoed this by adding to module AB an import of AB2 with a refinement mapping,
and checking `THEOREM Spec => AB2!Spec`. 

The expressions in that refinement mapping must be written in terms of the variables
of module AB, but that is impossible without adding imaginary variables to AB
that remember where messages were lost from `AtoB` and `BtoA`.

Imaginary variables
- do NOT need to describe any actual state of the system.
  - Actually, if they can be described in terms of the original system, 
    they are really unnecessary.
- do NOT need to be implemented
- MAY be needed to construct a refinement mapping

They are often called _auxiliary variables_.

## What's next ?

- Writing our own specs, including liveness conditions
- Showing that one spec implements another
- It won't be easy at first: 
  - writing good specs takes practice
  - reading other people's specs can help
- Discovering the remaining features of TLA+ in Lamport's book
- PlusCal

Remember to take the time to stop and think.