Skip to main content

Worksheet Problem Set M2

Instructions: You may type up or handwrite your work, but it must be neat, professional, and organized and it must be saved as a PDF file and uploaded to the appropriate Gradescope assignment. Use a scanner or scanning app to convert handwritten work on paper to PDF. I encourage you to type your work using the provided template.
All tasks below must have a complete solution that represents a good-faith attempt at being right to receive engagement credits. If your submission is complete and turned in on time, you will receive full engagement credits for the assignment. All other submissions will receive zero engagement credit. Read the guidelines at Grading Specifications carefully.
To abide by the class’ academic honesty policy, your work should represent your own understanding in your own words. If you work with other students, you must clearly indicate who you worked with in your submission. The same is true for using tools like generative AI although I strongly discourage you from using such tools since you need to build your own understanding here to do well on exams.

1. Building New Codes from Old.

Let \(C_1\) and \(C_2\) be linear codes of the same length \(n\) over the finite field \(\mathbb{F}_q\) and let \(G_1\) and \(G_2\) be generator matrices for \(C_1\) and \(C_2\text{,}\) respectively. Define the following codes:
Here \((\cdot\mid\cdot)\) denotes concatenation of vectors. For \(i=1,2,\ldots,6\) denote by \(k_i\) the dimension and \(d_i>\) the minimum distance of the code \(C_i\text{.}\) Assume that \(k_1,k_2\) are both greater than zero.

(a)

Show that \(C_3\) is linear if and only if either \(C_1\subseteq C_2\) or \(C_2\subseteq C_1\text{.}\)
Solution.
First, suppose that \(C_3\) is linear. Then, for any \(c_1 \in C_1\) and \(c_2 \in C_2\text{,}\) their sum \(c_1 + c_2\) must also be in \(C_3\text{.}\) Since \(C_3 = C_1 \cup C_2\text{,}\) this means that \(c_1 + c_2\) must be in either \(C_1\) or \(C_2\text{.}\) Without loss of generality, assume \(c_1 + c_2 \in C_1\text{.}\) Since \(C_1\) is linear, it follows that \(c_2 = (c_1 + c_2) - c_1 \in C_1\text{.}\) Therefore, \(C_2 \subseteq C_1\) since \(c_2\) was arbitrary.
Now suppose that either \(C_1 \subseteq C_2\) or \(C_2 \subseteq C_1\text{.}\) Then either \(C_3=C_2\) or \(C_3=C_1\text{,}\) both of which are linear codes. Thus, \(C_3\) is linear.

(b)

Show that the codes \(C_4\text{,}\) \(C_5\text{,}\) and \(C_6\) are linear.
Solution.
\(C_4\) is linear. We need to show that \(C_4\) is closed under addition and scalar multiplication. So let \(x,y\in C_4\) and \(a,b\in\mathbb{F}_q\text{.}\) Because \(C_4=C_1\cap C_2\text{,}\) we have \(x,y\in C_1\) and \(x,y \in C_2\text{.}\) Since both are linear it follows that \(ax+by\in C_1\) and \(ax+by\in C_2\text{,}\) so \(ax+by\in C_1\cap C_2=C_4\text{.}\) Thus, \(C_4\) is linear.
\(C_5\) is linear. We need to show that \(C_5\) is closed under addition and scalar multiplication. So let \(x,y\in C_5\) and \(a,b\in\mathbb{F}_q\text{.}\) Then we have elements \(x_1,y_1\in C_1\) and \(x_2,y_2\in C_2\) such that \(x=x_1+x_2\) and \(y=y_1+y_2\text{.}\) Therefore,
\begin{equation*} ax+by = a(x_1+x_2)+b(y_1+y_2) = (ax_1+by_1)+(ax_2+by_2) \in C_1 + C_2 = C_5 \end{equation*}
since both \(C_1\) and \(C_2\) are linear. Thus, \(C_5\) is linear.
\(C_6\) is linear. We need to show that \(C_6\) is closed under addition and scalar multiplication. So let \(x,y\in C_6\) and \(a,b\in\mathbb{F}_q\text{.}\) Now we have elements \(x_1,y_1\in C_1\) and \(x_2,y_2\in C_2\) such that \(x=(x_1\mid x_2)\) and \(y=(y_1\mid y_2)\text{.}\) Therefore,
\begin{equation*} ax+by = a(x_1\mid x_2)+b(y_1\mid y_2) = (ax_1+by_1 \mid ax_2+by_2) \in C_6 \end{equation*}
since both \(C_1\) and \(C_2\) are linear. Thus, \(C_6\) is linear.

(c)

Show that if \(k_4\gt 0\) then \(d_4 \geq \max\{d_1,d_2\}\)
Solution.
Suppose \(k_4\gt 0\text{.}\) Then there exist at least two codewords in \(C_4=C_1\cap C_2\) it has size \(q^k\geq 2^1\text{.}\) So let \(c_1,c_2 \in C_4\text{.}\) Now we also have \(c_1,c_2 \in C_1\) which means \(d(c_1,c_2)\geq d_1\) and similarly \(c_1,c_2 \in C_2\) so that \(d(c_2,c_2)\geq d_2\text{.}\) Thus \(d(c_1,c_2)\geq \max\{d_1,d_2\}\) and since \(c_1,c_2\) were arbitrary, it follows that \(d_4\geq \max\{d_1,d_2\}\text{.}\)

(d)

Show that \(k_5 \leq k_1+k_2\) with equality if and only if \(k_4=0\text{.}\)
Solution.
First, let \(B_1\) and \(B_2\) be bases for \(C_1\) and \(C_2\) respectively. Then \(B_5=B_1\cup B_2\) is a spanning set for \(C_5=C_1+C_2\text{,}\) so
\begin{equation*} k_5 \leq |B_5| \leq |B_1|+|B_2| = k_1+k_2\text{.} \end{equation*}
Now, suppose we have \(k_5=k_1+k_2\text{.}\) Then both inequalities above must be strict, so that \(B_5\) is a basis and not just a spanning set for \(C_5\) and \(B_1\) and \(B_2\) are disjoint. Now let \(x\in C_1\cap C_2\text{.}\) Then we may write \(x\) as a linear combination of the vectors in \(B_1\) and also as a linear combination of the vectors in \(B_2\text{.}\) Equating these two expressions for \(x\) and rearranging gives a linear combination of the vectors in \(B_5\) which is \(0\text{,}\) which means we must have all coefficients equal to \(0\) since \(B_5\) is a basis. Thus, \(x=0\) and so \(C_1\cap C_2=\{0\}\) and \(k_4=0\text{.}\)
On the other hand, suppose \(k_4=0\) and so \(C_1\cap C_2=\{0\}\text{.}\) We now need to show that \(B_5\) is a basis for \(C_5\text{.}\) We already know it is a spanning set, so we just need to show it is linearly independent. Suppose we have a linear combination of the vectors in \(B_5\) that equals \(0\text{:}\)
\begin{equation*} \sum_{i=0}^{k_1} \lambda_i b_{1i} + \sum_{j=0}^{k_2} \mu_j b_{2j} = 0\text{,} \end{equation*}
where the \(b_{1i}\) are the vectors in \(B_1\) and the \(b_{2j}\) are the vectors in \(B_2\text{.}\) Rearranging gives
\begin{equation*} \sum_{i=0}^{k_1} \lambda_i b_{1i} = -\sum_{j=0}^{k_2} \mu_j b_{2j} \end{equation*}
so this vector is in both \(C_1\) and \(C_2\text{.}\) Since the only vector in both is \(0\text{,}\) we get that both sums are equal to \(0\text{.}\) Since \(B_1\) and \(B_2\) are bases, it follows that all the coefficients must be zero, so the linear combination is trivial and \(B_5\) is linearly independent. Thus, \(B_5\) is a basis for \(C_5\) and so \(k_5=k_1+k_2\text{.}\)

(e)

Show that \(d_5\leq \min\{d_1,d_2\}\text{.}\)
Solution.
Since \(C_1\subseteq C_5\) and \(C_2\subseteq C_5\text{,}\) we have \(d_5 \leq d_1\) and \(d_5 \leq d_2\text{,}\) so \(d_5 \leq \min\{d_1,d_2\}\text{.}\)

(f)

[Optional]. Show that
\begin{equation*} \left(\begin{array}{c|c} G_1 \amp 0 \\ \hline 0 \amp G_2 \end{array}\right) \end{equation*}
is a generator matrix of \(C_6\) and so \(k_6 = k_1 + k_2\text{.}\)
Solution.
This is the solution.

2. \(q\)-ary Hamming codes.

In class we introduced the \(q\)-ary Hamming codes as the linear codes over \(\mathbb{F}_q\) with parity check matrix \(H_{q,r}\) where the columns are all of the nonzero vectors of \(\mathbb{F}_q^r\) that have as their first nonzero entry a \(1\text{.}\) Justify our claim that these codes are in fact \(\left[ \frac{q^r-1}{q-1}, \frac{q^r-1}{q-1}-r,3\right]_q\) codes.
Solution.
Length. There are \(q^r-1\) nonzero vectors in \(\mathbb{F}_q^r\text{,}\) but the restriction on the first nonzero entry means that we only have one choice instead of \(q-1\) for the first nonzero entry, so there are \(\frac{q^r-1}{q-1}\) columns in the parity-check matrix and thus the code has length \(\frac{q^r-1}{q-1}\text{.}\)
Dimension. Because the columns of the parity-check matrix include all nonzero vectors of \(\F_q^r\) with first entry \(1\text{,}\) in particular the columns include all \(r\) of the standard basis vectors for \(\F_q^r\text{,}\) so the parity-check matrix has rank \(r\) and thus the code has dimension \(k=n-\rk(H)=\frac{q^r-1}{q-1}-r\text{.}\)
Distance. Since the columns of \(H_{q,r}\) all have the same first entry and different later entries, they are not scalar multiples of each other, so they are pairwise linearly independent. Further none is the zero vector, so the distance is at least \(3\text{.}\) On the other hand, the distance is at most \(3\) since for example the columns \((1,0,0,\ldots,0)\text{,}\) \((0,1,0,\ldots,0)\text{,}\) and \((1,1,0,\ldots,0)\) are dependent. Thus the distance is exactly \(3\text{.}\)

3. Ambiguous Decoding.

Let \(C\) be the linear code over \(\F_3\) generated by
\begin{equation*} G=\begin{bmatrix} 2 \amp 1 \amp 2 \amp 1 \\ 1 \amp 1 \amp 1 \amp 0 \end{bmatrix}\text{.} \end{equation*}

(a)

List all the codewords of \(C\text{.}\)
Solution.
The codewords are \(\{0000, 2121, 1212, 1110, 2220, 0201, 0102, 1011,2022\}\text{.}\)

(b)

Find a systematic generator matrix of \(C\text{.}\)
Solution.
By applying row operations until we get to the reduced row echelon form, we get the following systematic generator matrix for \(C\text{:}\)
\begin{equation*} \tilde{G}=\begin{bmatrix} 1 \amp 0 \amp 1 \amp 1 \\ 0 \amp 1 \amp 0 \amp 2 \\ \end{bmatrix} \end{equation*}

(c)

Compute the minimum distance of \(C\text{.}\)
Solution.
We could compute the parity-check matrix and examine dependencies in its columns, since we have a systematic generator, but it is quicker to glance at the listed codewords and see that the minimum nonzero weight is 2 (for \(0102,0201\)) so the minimum distance is 2.

(d)

A codeword of \(C\) is transmitted through a noisy channel and the word \(\mathbf{y}=(1,1,1,1)\) is received. Find all the codewords of \(C\) that could be produced as an output by a nearest-codeword decoder for \(C\) when applied to \(\mathbf{y}\text{.}\)
Solution.
The two nearest codewords to \(\by\) are \(1110\) and \(1011\text{,}\) both of which are distance 1 from \(\by\text{.}\) Either could be produced by a nearest-codeword decoder.

(e)

Suppose encoding is carried about by mapping an information word \(\mathbf{u} \mapsto \mathbf{u}G \text{.}\) A nearest-codeword decoder is applied to the received word \(\mathbf{y}\) from the previous part to produce a codeword \(\hat{\mathbf{c}}\text{.}\) Denote by \(\hat{\mathbf{u}}\) the information word that produces \(\hat{\mathbf{c}}\) under the encoding map. Are there any entries of \(\hat{\mathbf{u}}\) that are uniquely determined by \(\mathbf{y}\) regardless of which nearest codeword \(\hat{\mathbf{c}}\) is selected by the decoder?
Solution.
The message vectors \(\hat{\bu}_1\) and \(\hat{\bu}_2\) producing these codewords are \(01\) and \(12\) respectively. These differ in both positions, so no entries of the message vector are uniquely determined by the received word in this case.

(f)

Repeat the previous part for the case where the systematic generator matrix found in part (b) is used for encoding instead.
Solution.
When we use the systematic generator for encoding, the message vectors producing the two nearest codewords are \(11\) and \(10\text{,}\) which differ in the second position, so the first entry of the message vector is uniquely determined by the received word in this case. This is an advantage of using a systematic generator for encoding for this code since it allows us to decode at least part of the message even when there are multiple nearest codewords.

4. Step-by-step Decoding for Linear Codes.

Consider the binary code generated by the matrix
\begin{equation*} G=\begin{bmatrix} 1 \amp 0 \amp 0 \amp 1 \amp 1 \amp 0 \\ 0 \amp 1 \amp 0 \amp 0 \amp 1 \amp 1 \\ 0 \amp 0 \amp 1 \amp 1 \amp 0 \amp 1 \end{bmatrix}\text{.} \end{equation*}
In this problem, you will use the algorithm above to do decoding for this code.

(a)

Construct a 1-to-1 correspondence between syndromes and weights of the lightest vectors in the corresponding cosets for this code.
Solution.
The syndromes and weights of the lightest vectors in the corresponding cosets are as follows (using the parity-check matrix given below).
\begin{equation*} H=\begin{bmatrix} 1 \amp 0 \amp 1 \amp 1 \amp 0 \amp 0 \\ 1 \amp 1 \amp 0 \amp 0 \amp 1 \amp 0 \\ 0 \amp 1 \amp 1 \amp 0 \amp 0 \amp 1 \end{bmatrix} \end{equation*}
Coset Leader Syndrome Weight
000000 000 0
100000 110 1
010000 011 1
001000 101 1
000100 100 1
000010 010 1
000001 001 1
001010 111 2

(b)

Decode the following received vectors using step-by-step decoding:
(i)
\(\mathbf{y}_1=(100010)\)
Solution.
The syndrome of \(\mathbf{y}_1\) is \(100\) corresponding to a coset leader of weight 1. Now we check \(\by_1+\be_1=000010\text{.}\) This has syndrome \(010\) corresponding to a coset leader of weight 1, which is the same as the weight of the original syndrome. So we do not update \(\by_1\) and return to the top of the algorithm.
Next we check \(\by_1+\be_2=110010\text{.}\) This has syndrome \(111\) corresponding to a coset leader of weight 2, which is larger than the weight of the original syndrome. So we do not update \(\by_1\) and return to the top of the algorithm.
Next we check \(\by_1+\be_3=101010\text{.}\) This has syndrome \(001\) corresponding to a coset leader of weight 1, which is the same as the weight of the original syndrome. So we do not update \(\by_1\) and return to the top of the algorithm.
Next we check \(\by_1+\be_4=100110\text{.}\) This has syndrome \(000\) corresponding to a coset leader of weight 0, which is less than the weight of the original syndrome. So we update \(\by_1\coloneqq \by_1+\be_4=100110\) and since the syndrome has weight 0 we stop and output \(100110\) as the transmitted codeword.
(ii)
\(\mathbf{y}_2=(100001)\)
Solution.
The syndrome of \(\mathbf{y}_2\) is \(111\) corresponding to a coset leader of weight 2. Now we check \(\by_2+\be_1=000001\text{.}\) This has syndrome \(001\) corresponding to a coset leader of weight 1, which is less than the weight of the original syndrome. So we update \(\by_2\coloneqq \by_2+\be_1\) and return to the top of the algorithm.
Next we check \(\by_2+\be_2\text{.}\) This has syndrome \(010\) corresponding to a coset leader of weight 1, which is the same as the weight of the updated syndrome. So we do not update \(\by_2\) and return to the top of the algorithm.
Next we check \(\by_2+\be_3\text{.}\) This has syndrome \(100\) corresponding to a coset leader of weight 1, which is the same as the weight of the updated syndrome. So we do not update \(\by_2\) and return to the top of the algorithm.
Next we check \(\by_2+\be_4\text{.}\) This has syndrome \(101\) corresponding to a coset leader of weight 1, which is the same as the weight of the updated syndrome. So we do not update \(\by_2\) and return to the top of the algorithm.
Next we check \(\by_2+\be_5\text{.}\) This has syndrome \(011\) corresponding to a coset leader of weight 1, which is the same as the weight of the updated syndrome. So we do not update \(\by_2\) and return to the top of the algorithm.
Next we check \(\by_2+\be_6\text{.}\) This has syndrome \(000\) corresponding to a coset leader of weight 0, which is less than the weight of the original syndrome. So we update \(\by_2\coloneqq \by_2+\be_6=000000\) and since the syndrome has weight 0 we stop and output \(000000\) as the transmitted codeword.
(iii)
\(\mathbf{y}_3=(010100)\)
Solution.
The syndrome of \(\mathbf{y}_3\) is \(111\) corresponding to a coset leader of weight 2 again. Now we check \(\by_3+\be_1=110100\text{.}\) This has syndrome \(001\) corresponding to a coset leader of weight 1, which is less than the weight of the original syndrome. So we update \(\by_3\coloneqq \by_3+\be_1\) and return to the top of the algorithm.
Next we check \(\by_3+\be_2\text{.}\) This has syndrome \(010\) corresponding to a coset leader of weight 1, which is the same as the weight of the updated syndrome. So we do not update \(\by_3\) and return to the top of the algorithm.
Next we check \(\by_3+\be_3\text{.}\) This has syndrome \(100\) corresponding to a coset leader of weight 1, which is the same as the weight of the updated syndrome. So we do not update \(\by_3\) and return to the top of the algorithm.
Next we check \(\by_3+\be_4\text{.}\) This has syndrome \(101\) corresponding to a coset leader of weight 1, which is the same as the weight of the updated syndrome. So we do not update \(\by_3\) and return to the top of the algorithm.
Next we check \(\by_3+\be_5\text{.}\) This has syndrome \(011\) corresponding to a coset leader of weight 1, which is the same as the weight of the updated syndrome. So we do not update \(\by_3\) and return to the top of the algorithm.
Next we check \(\by_3+\be_6\text{.}\) This has syndrome \(000\) corresponding to a coset leader of weight 0, which is less than the weight of the original syndrome. So we update \(\by_3\coloneqq \by_3+\be_6=110101\) and since the syndrome has weight 0 we stop and output \(110101\) as the transmitted codeword.
(iv)
\(\mathbf{y}_4=(111100)\)
Solution.
The syndrome of \(\mathbf{y}_1\) is \(100\) corresponding to a coset leader of weight 1. Now we check \(\by_1+\be_1=011100\text{.}\) This has syndrome \(010\) corresponding to a coset leader of weight 1, which is the same as the weight of the original syndrome. So we do not update \(\by_1\) and return to the top of the algorithm.
Next we check \(\by_1+\be_2\text{.}\) This has syndrome \(111\) corresponding to a coset leader of weight 2, which is larger than the weight of the original syndrome. So we do not update \(\by_1\) and return to the top of the algorithm.
Next we check \(\by_1+\be_3\text{.}\) This has syndrome \(001\) corresponding to a coset leader of weight 1, which is the same as the weight of the original syndrome. So we do not update \(\by_1\) and return to the top of the algorithm.
Next we check \(\by_1+\be_4\text{.}\) This has syndrome \(000\) corresponding to the coset leader \(000000\) of weight 0, which is less than the weight of the original syndrome. So we update \(\by_1\coloneqq \by_1+\be_4=111000\) and since the syndrome has weight 0 we stop and output \(111000\) as the transmitted codeword.

(c)

Which of the vectors in (b) were decoded to a unique closest codeword?
Solution.
\(\by_1\) and \(\by_4\) were decoded to a unique closest codeword since they belong to cosets with a unique minimum weight vector. The other two messages both belong to the coset with two weight 2 leaders, so they were not decoded to a unique closest codeword. Each has at least two codewords that are closest to it.

(d)

Describe in words what the algorithm you employed is doing as you process the received vector.
Solution.
The algorithm is checking if the received vector is a codeword. If it is not, it checks if flipping each bit of the received vector produces a vector that is closer to a codeword than the original received vector. If so, it updates the received vector by flipping that bit and then repeats this process until it finds a codeword.

5. First order Reed-Muller codes [Optional].

Let \(q\) be a prime power, \(m\) be a positive integer, and \(n=q^m\text{.}\) The first-order Reed-Muller code over \(\mathbb{F}_q\) is the linear \([n,m+1]\) code \(\mathrm{RM}(1,m)_q\) over \(\mathbb{F}_q\) with an \((m+1)\times n\) generator matrix whose columns range over all the vectors in \(\mathbb{F}_q^{m+1}\) with a first entry equaling 1.
Note that when \(q=2\) this is the dual code of the extended Hamming code defined by adding an overall parity bit as the first entry of every codeword.

(a)

Give the generator matrix of the first-order Reed-Muller code over \(\mathbb{F}_3\) with \(m=2\text{.}\)
Solution.
This is the solution.

(b)

Show that the minimum distance of \(\mathrm{RM}(1,m)_q\) is \(q^{m-1}(q-1)\) and that this number is the Hamming weight of \(q(q^m-1)\) codewords in \(\mathrm{RM}(1,m)_q\text{.}\) What are the Hamming weights of the remaining \(q\) codewords in \(\mathrm{RM}(1,m)_q\text{?}\)
Hint.
You may find it helpful to use the fact that for any nonzero vector \(\mathbf{a}=(a_1 a_2 \ldots a_n) \in \F_q^n\) the map \(f:\F_q^n \to \F_q\) defined by
\begin{equation*} f(\mathbf{x}) = \mathbf{a}\cdot\mathbf{x} \end{equation*}
is a surjective (onto) map and is \(q^{n-1}\)-to-1 (so each element of \(\F_q\) is the image of exactly \(q^{n-1}\) elements of \(\F_q^n\)).
Solution.
This is the solution.