I work in quantum error correction, and was trying to collect interesting and quantitative examples of repetition codes being used implicitly in classical systems. Stuff like DRAM storing a 0 or 1 via the presence or absence of 40K electrons [1], undersea cables sending X photons per bit (don't know that one yet), some kind of number for a transistor switching (haven't even decided on the number for that one yet), etc.

A key reason quantum computing is so hard is that by default repetition makes things worse instead of better, because every repetition is another chance for an unintended measurement. So protecting a qubit tends to require special physical properties, like the energy gap of a superconductor, or complex error correction strategies like surface codes. A surface code can easily use 1000 physical qubits to store 1 logical qubit [2], and I wanted to contrast that with the sizes of implicit repetition codes used in classical computing.

1: https://web.mit.edu/rec/www/dramfaq/DRAMFAQ.html

2: https://arxiv.org/abs/1208.0928

Regarding the sensitivity of Subsea systems they are still significantly above 1 photon/bit, the highest sensitivity experiments have been done for optical space comms (look e.g. for the work from Mit Lincoln Labs, David Geisler, David Kaplan and Bryan Robinson are some of the people to look for.

Repetition codes have a very clearly defined meaning in communication theory, using them to mean something else is very confusing.

All classical phenomena are repetition codes (e.g., https://arxiv.org/abs/0903.5082 ). And this is perfectly compatible with the meaning in communication theory, except that the symbols we're talking about are the states of the fundamental physical degrees of freedom.

In the exact same sense, the von Neumann entropy of a density matrix is the Shannon entropy of its spectrum, and no one says "we shouldn't call that the Shannon entropy because Shannon originally intended to apply it to macroscopic signals on a communication line".

As we build systems that require less power, we are also building systems that use less electrons to represent a single bit.

Another fun tidbit in this space comes down to signal propagation in a lossy medium: we don’t actually use square waves for our clocks since a square wave is composed of many different frequencies. Since different frequencies propagate at different speeds, a square wave looks less and less like a square and also uses more power than a simple sine wave. If you remember your Fourier/Laplace knowledge, you probably remember that a sine is a single frequency and thus it will be coherent through a conductor and use the least amount of power to generate.

Edit: I’m talking about electrons here but the concept of many packets for a single state extends to most of our communication channels today … eg radio communications like we see to/from Voyager.

Some people suggest that digital computing and neural networks are a bit fit, and that would should be using analog devices.

That sounds very appealing at first. But we have (at least) two problems:

First, our transistors dissipate almost no energy when they are either 'fully open' or 'fully closed'. Because either there's approximately no current, or approximately no resistance. Holding them partially open, like you'd do in analog processing, would produce a lot of heat.

The second problem: electrons are discrete, and thanks to miniaturisation and faster and faster clockspeeds, we are actually getting into realms where that makes a difference. So either you have to accept that the maximum resolution of activation of your analog neuron is fairly small (perhaps 10 bits or so?), which is not that much better than using your transistors in binary only; or you'll have to use much larger transistors in your neural chips.

Both problems together mean that analog computing for neural networks isn't really competitive with digital computing. (Outside of some very niche applications, perhaps.)

Even if the world were fundamentally classical (which it's not), the only things you could actually know about it would be the tiny subset that is amplified to macroscopic scales, necessarily producing many records in, if nothing else, the many atoms in the neurons in your brain.

> This insight captures the essence of Quantum Darwin- ism: Only states that produce multiple informational off- spring – multiple imprints on the environment – can be found out from small fragments of E. The origin of the emergent classicality is then not just survival of the fittest states (the idea already captured by einselection), but their ability to “procreate”, to deposit multiple records – copies of themselves – throughout E

Basically, quantum information is unknown. All possible classical states are mixed together. Once you measure it, it decoheres into a classical system, where all measurements are highly correlated -- some states exist, and others don't exist. A quantum coin flip is heads or tails on top, and tails or heads on bottom. But once you measure it, it all decoheres at once. You get a classical coin flip, that is hrads on top and tails on bottom, or vice versa. You can't get heads on top, and tails or heads on bottom unsettled for a second bit of information. The same message is written twice, on the top and the bottom of the coin. You can look at the top of coin or the bottom and get the same result. If the top of the coin melts, you can still read the result of the flip from the bottom.

Eg Google stores data internally with something like Reed-Solomon error correction, but they typically have two independent copies.

So they have repetition code at the classical nano-scale, then Reed-Solomon error correction at the next level, and at the highest level they apply repetition again.

There's nothing confusing about this, as long as you are careful to make sure that your listener knows which level you are talking about.

> Repetition codes have a very clearly defined meaning in communication theory, using them to mean something else is very confusing.

OP used them exactly with the orthodox meaning as far as I can tell.

I'd assume that these days it's a couple of orders of magnitude fewer than that (the cited source is from 1996). Incidentally, 40k e- is roughly the capacity of a single electron well ("pixel") in a modern CMOS image sensor [1] – but those 40k electrons are able to represent a signal of up to ~14 bits, around 10k distinct luminance values, depending on temperature and other noise sources.

[1] https://www.princetoninstruments.com/learn/camera-fundamenta...

In general, I'm not sure that the classical information theory toolkit allows us to compare a coherent state with some average occupation number N to say, M (not necessarily coherent) states with average occupation number N' such that N' * M = N. For example, you could use a state that is definitely not "classical" / a coherent state or you could use photon number resolving measurements.

A tangential remark: The classical information theory field uses this notion of "energy per bit" to be able to compare more universally between information transmission schemes. So they would ask something like "How many bits can I transmit with X bandwidth and Y transmission power?"

Yes. But regardless of whether its feasible to send single quanta in any given circumstance, the redundant nature of the signals is key to understanding its much higher degree of robustness relative to quantum signals.

And to be clear, you can absolutely send a classical signal with individual quanta.

Yes. Though how 'classical' your understanding of that system is stands to reason?

Can you elaborate on this a bit? My intuition is that, by default, statistical models benefit from larger N. But I have no experience in quantum physics.

There are ways of repeating quantum information that protect against accidental measurement errors. For example, if your logical 0 is |000> + |110> + |011> + |101> and your logical 1 is |111> + |001> + |100> + |010> then can recover from one accidental measurement. And there are more complex states that protect against both bitflip errors and accidental measurements simultaneously. They're just more complicated to describe (and implement!) than "use 0000000 instead of 0 and 1111111 instead of 1".

Classical systems: You measure some state, with the measurement containing some error. Averaging the measurement error usually gets closer to the actual value.

Quantum systems: Your measurement influences/can influence the state, which can cause an error in the state itself. Multiple measurements means more possible influence.

Even dispersion is a boring question because it is possible to reverse dispersion by sending the light through a parametric amplifier to conjugate the phases and then running it through the dispersion medium a second time locally.

We later ended up working on other things.

if we consider "quantum" to mean our quantum theory, at the level of general relativity, gravity is not a quantum system. and the qualifier "yet" is also not known.

But quantum field actually is a universe model like GR and hence you have this issue of sort of action in a distance by superimposition.

Hence zoom in is not helping.

Thanks for asking it. Thanks too to the person that provided the thorough answer.

After reading that answer - seeing all the math equations and physics that cover several disciplines - I wonder how some people can just hand-wave "science" away as a conspiracy to fool the masses. They clearly have little idea the amount of knowledge that works together to get answers questions like this.

wouldn't you also want to know how many photons are transmitted and how many bits transmitted are received?

To beat Shannon you need PPM formats and photon counters (single photon detectors).

One can do significantly better than the numbers from voyager in the article using optics even without photon cpunting. Our group has shown 1 photon/bit at 10 Gbit/s [1] but others have shown even higher sensitivity (albeit at much lower data rates).

[1] https://www.nature.com/articles/s41377-020-00389-2

For example, MIMO appears to "break" the Shannon-Hartley limit because it does exceed the theoretical AWGN capacity for a simple channel. However, when you apply Shannon's theory to reformulate the problem for the case of a multipath channel with defined mutual coupling, you find that there is a higher limit you are still bounded by.

They send optical pulses in one of up to 128 possible time slots, thereby carrying 7 bits each. And each optical pulse on earth may only be received by 5-10 photons.

You are right that you can randomly lose some photons. That's what error correcting codes are. See https://en.wikipedia.org/wiki/Error_correction_code

As an example, assume every photon can encode 10 bits without losses, but you lose 10% of your photons. Then with a clever error correcting code you can encode just shy of 9 bits per photon.

You can think of the error correcting code 'smearing' 9 * n bits of information over 10 * n photos, and as long as you collect 0.9 * n photons, you can recover the 9 * n bits of information.

It's the same reason your CD still plays, even if you scratch it. In fact, you can glue a paper strip of about 1 cm width on the bottom of your CD, and it'll still play just fine. Go wider, and it won't, because you'll be exceeding the error correcting capacity of the code they are using for CDs.

"Photon-counting" methods cannot be implemented at frequencies so low as used in 5G networks or in any other traditional radio communications.

Higher frequencies can carry more data as you infer but the engineering challenges of designing transmitters and receivers create tradeoffs in practical systems.

If you have a way to reliably transmit N bits in time T using P photons, you can transmit N+1 bits in time 2 * T using also P photons. What you would do to transmit X0,X1,...Xn is:

- During the first time slot of duration T, transmit X1,... Xn if X0 = 0 and 0 otherwise (assuming absence of photons is one of the symbols, which we can label 0)

- During the second time slot of duration T, transmit 0 if X0=0 and X1,... Xn otherwise

This only uses P photons to transmit one more bit, but it takes twice as long. So if you're allowed to take all the time that you want to transmit, and have really good clocks, I guess that theoretically this is unbounded.

It seems there might be multiple ways to go beyond Shannon’s limit, depending on what you are trying to do

> Around 2004, Emmanuel Candès, Justin Romberg, Terence Tao, and David Donoho proved that given knowledge about a signal's sparsity, the signal may be reconstructed with even fewer samples than the sampling theorem requires.[4][5] This idea is the basis of compressed sensing

…

> However, if further restrictions are imposed on the signal, then the Nyquist criterion may no longer be a necessary condition. A non-trivial example of exploiting extra assumptions about the signal is given by the recent field of compressed sensing, which allows for full reconstruction with a sub-Nyquist sampling rate. Specifically, this applies to signals that are sparse (or compressible) in some domain

From: https://en.m.wikipedia.org/wiki/Nyquist%E2%80%93Shannon_samp...

Unlike the Nyquist-Shannon theory, compressed sensing is not generally applicable: it requires a sparse signal.

As with many other optimization techniques, it’s a trade off between soundness and completeness.

https://doi.org/10.1109%2FJRPROC.1962.288169

I’m wondering: would a probe launched today instead employ a laser to communicate? This would seem to offer many orders of magnitude improvement in the directionality of the signal.

However, due to the shape of the black body radiation curve, the sun gives out relatively less microwave radiation than it does visible light, which might outweigh the advantages of more directionality given by using a laser.

Not having thought this through before, I see now that while a transmit maser may have efficiency advantages, it may not improve directionality relative to a standard parabolic radio transmitter. All methods of producing microwaves will have basically the same diffraction-limited gain for a given “aperture” (dish) size. That darn uncertainty principle! (However, an optical laser would still give way better directionality.)

We want to download high resolution images/spectrographs whereas we only want to upload code/instructions.

Probably infeasible for several reasons (only useful when accelerating DIRECTLY away from Earth, incoming light to power spaceship is probably coming from the sun and therefore likely also in the directly of Earth, so net zero acceleration at best from firing the photons back towards the sun), but it'd be pretty neat.

My intuition would have been that you are better off using a fairly standard transceiver and spending your engineering budget either increasing power or getting a bigger dish (either by launching on a wider rocket or with a folding design).

Lasers might interesting for the downlink, but receiving a laser signal on the probe sounds difficult (earth is pretty bright).

https://www.youtube.com/watch?v=ExhSqq1jysg

Not that this changes anything, we can only detect or create light with matter. but it does make me curious about single photon experiments and what they are actually measuring.

Light follows a null geodesic through space-time with zero (space-time) length and no proper time. Past, future, and causality have no meaning to a photon. We think of photons travelling through space because our symmetry is broken, we have mass, and we and experience time and space.

Observers like us will see light follow the same world-line from its source to its target. It cannot interact with anything else, and some might say it was only ever emitted so that it could interact with its target.

So from a certain point of view the “existence” of a photon is entirely bound up in its interaction with source and target, and it’s not really useful to speak of it in other terms. The quantised interaction is the photon.

For the "how many photons are needed" question, I agree that 320 baud (i.e., the effective analog bandwidth of 320 Hz) should have been used for the Shannon-Hartley calculations.

As to how they are decoded, you'll need to understand some modulation techniques.

So 1500 Photons hit the receiver per bit send, but this is obviously way to few to keep processing the signal and it will just be drowned out by noise? Where do we go from here? Does voyager repeat its signal gazillions of times so we can average out the noise on our end? Where can I find more information on what is done with these few photons?

And can we beat the Shannon limit somehow, eg collect for longer, put the dish outside the atmosphere, and so on?

1. Increase the effective receiving dish size, to capture more of the signal. Essentially, this would be effective in direct proportion to beam spread (the more beam spread, the bigger dish you can use to capture signal).

In practice, this would use multiple geographically-displaced dishes to construct a virtually-larger dish, to allow for better noise-cancellation magic (and at lower cost than one huge dish). I believe the deep space network (DSN) already does this? Edit: It certainly has arrayed antennae [0], though not sure how many are Voyager-tasked.

2. Increase the resilience of the signal, via encoding. The math is talking about bits and photons, but not encoded information. By trading lower bit-efficiency for increased error tolerance (i.e. including redundant information) we can extract a coherent signal even accounting for losses.

Someone please point out if I'm wrong, but afaik the Shannon–Hartley limit speaks to "lower" in the physical stack than error coding. I.e. one can layer arbitrary error coding on top of it to push limits (at the expense of rate)?

If the above understanding is correct, is there a way to calculate maximum signal distance assuming a theoretically maximally efficient error coding (is that a thing?) ? Or is that distance effectively infinite, assuming you're willing to accept an increasingly slow bit receiving rate?

[0] https://en.m.wikipedia.org/wiki/NASA_Deep_Space_Network#Ante...

https://www.allaboutcircuits.com/news/voyager-mission-annive...

> The uplink carrier frequency of Voyager 1 is 2114.676697 MHz and 2113.312500 MHz for Voyager 2. The uplink carrier can be modulated with command and/or ranging data. Commands are 16-bps, Manchester-encoded, biphase-modulated onto a 512 HZ square wave subcarrier.

The "Manchester encoding" brings us to https://www.allaboutcircuits.com/technical-articles/manchest...

https://en.wikipedia.org/wiki/Manchester_code

Note that "16 bps" while the system runs at 160 bps. This suggests that the data is repeated ten times and xor'ed with a clock running at 10 HZ.

While there's no VOY set up now, https://eyes.nasa.gov/dsn/dsn.html will occasionally show it. When that happens, you will likely see two set up for it. I've not seen them set up across multiple facilities - the facilities are 120° apart and only one has a spacecraft above the horizon for any given length of time.

---

In the "sensitivity to photons" category, I'll also mention https://en.wikipedia.org/wiki/Lunar_Laser_Ranging_experiment...

At the Moon's surface, the beam is about 6.5 kilometers (4.0 mi) wide[24][i] and scientists liken the task of aiming the beam to using a rifle to hit a moving dime 3 kilometers (1.9 mi) away. The reflected light is too weak to see with the human eye. Out of a pulse of 3×10^17 photons aimed at the reflector, only about 1–5 are received back on Earth, even under good conditions. They can be identified as originating from the laser because the laser is highly monochromatic.

While there's no signal there, we're still looking at very sensitive equipment.

I'm no physicist here so take this with a major grain of salt. I think the limit might ultimately arise from the uncertainty principle? Eventually the signal becomes so weak that measuring it, overwhelms the signal. This is why the receiver of space telescopes is cooled down with liquid helium. The thermally-generated background RF noise (black bodies radiate right down into the radio spectrum) would drown everything else out otherwise.

Along those lines, while I'm still not quite sure where the limit is, things become discrete at the micro level, and the smallest possible physical state change appears to be discrete in nature: https://en.wikipedia.org/wiki/Landauer%27s_principle Enough work physically must occur to induce a state change of some kind at the receiver, or no communication can occur. (But this interpretation is disputed!)

There are practical signals we use every day that are "below the noise floor" before we decode them.

So while there is an ultimate limit of the maximum coding rate for a given signal-to-noise ratio, this is expressed in terms of a data rate (i.e. bits per second). If you're fine with lowering your data rate, there is no fundamental theoretical limit, as far as I understand.

You can't work around the Shannon limit by using encoding. It's the theoretical information content limit. But you can keep reducing the bandwidth and one way of doing that is adding error correction. So intuitively I'd say yes to your question, the distance can go to infinity as long as you're willing to accept an increasingly low receive bit rate. What's less clear to me is whether error correction on its own can be used to approach the Shannon limit for a given S/N ratio - I think the answer is no because you're not able to use the entire underlying bandwidth. But you can still extract a digital signal from noise given enough of a signal...

EDIT: There is a generalization of the Shannon limit to non-white Gaussian noise here: https://dsp.stackexchange.com/a/82840

Of course that would mean sending (a) giant receiver dish(es) in the general direction a probe is sent. On the flip side, if using a single relay it could travel at roughly 1/2 the speed of the probe.

Note that signal strength weakens with distance^2. So if eg. you'd have 2 relays (1/3 and 2/3 between Earth & the probe), each relay would receive 9x stronger signal.

No doubt the 'logistics' (trajectory, gravity assist options, mission cost etc) make this impractical. But it is an option.

There's some details on the Voyager gravity-assist mechanics here [0], but you'd also need the escape trajectory to be pointed in the Voyager direction which would further constrain...

That said, Earth-Jupiter-Saturn alignments don't seem that rare (on a decades scale).

[0] http://www.gravityassist.com/IAF3-2/Ref.%203-143.pdf

With each doubling of distance the number decreases by the inverse square law, so with the current setup we'd have a maximum distance of log2(60) = 5.9 times the current distance (about 163 astronomical units (AU)) which is 961 AU.

In comparison, the closest star to our sun, Proxima Centauri, is 268774 AU away!

Which means we would need something 268774/961 ~= 280 times SQUARED = 78222 times more sensitive than the current setup at the Shannon limit to communicate with it if it managed to get that far.

1) A bigger dish is the most obvious one.

2) Use a lower bitrate to send more energy per bit at the same transmitter power.

3) Reduce the effective receiver temperature. This is a case where putting it outside the atmosphere might help in reducing noise.

In fact, I assume that at this distance, even a very narrow signal would spread wide enough to illuminate more than just the Earth diameter.

I think the practical limit right now is that the Voyagers are losing power.

"The radioisotope thermoelectric generator on each spacecraft puts out 4 watts less each year. [...] The two Voyager spacecraft could remain in the range of the Deep Space Network through about 2036, depending on how much power the spacecraft still have to transmit a signal back to Earth."

https://voyager.jpl.nasa.gov/frequently-asked-questions/

If you turn on a faucet for longer, you're not beating the "gallon limit" of the system. The limit is not a fixed number, it's directly based on how much you do to improve the signal.

And they have already slowed down the transmission speed repeatedly.

(I recall seeing a video on that dish, and the director seemed confident there was enough noise margin left that voyager's power would fail before they lost contact with it)

They are closer, but the radar equation received power is inversely proportional to range to the fourth power, not range squared as with Voyager.

Anything proportional to 1/R^4 degrades very quickly.

If you’re curious about how many bits a single photon can carry, in controlled settings (tabletop quantum optics) a single photon can carry log(n) bits where n is the size of the state space of the photon, which theoretically is infinite and in practice it can reach into the hundreds/thousands.

Eg send a continuous wave with low signal beyond its phase, and measure at a rate of (digital) bits per month?

This makes me wonder, are the bits = the power turned on for exactly 1/320th sec, every 1/160th sec? Or is the power on/power off ratio something different? Does it vary by protocol? What are the pros and cons?

At least for Voyager->earth we can use giant radio telescopes to detect the faint signal, but how do we manage to focus on those few hundreds of photons per bit coming from a pinpoint source a light day away?!

In the earth->Voyager direction it seems even less intuitive - sure we can broadcast a powerful signal, but it's being received by a 12' wide antenna 15 billion miles away. WTF?

I guess radio communications in general is magic, a bit like (in nature of counter-intuition) quantum entanglement of particles arbitrarily far apart. It seems there is something deeply wrong about our mental models of space and time.

Equally we're broadcasting to the area of space Voyager's in. We're not able to to target it to the dish - 12' isn't the DSOC accuracy - it's the size required to pick up enough data given the signal diffusion.

This article is for Voyager 2, but the issue is the same. For a brief moment every year we actually get closer to Voyager 1, then we pivot away in our revolution around the sun and the distance between Earth and Voyager 1 or 2 increases sharply. So distance, when plotted over time, looks like a wobbly line.

https://earthsky.org/space/voyager-spacecraft-getting-closer...

In addition to it being 2.4GHz, this is the reason for having the National Radio Quiet Zone near Green Bank

If the frequency is high enough then the waves of light can be detected by things as small as cells in the back of your eye, or the pixels in a camera sensor. If it is too low then you need much larger detectors.

Other animals have detectors for different frequencies/wavelengths, allowing them to see either infra-red (mosquitos) or ultraviolet (bees, butterflies etc).

What we call "visible light" is just the particular range that our eyes can detect (about 400 to 800THz). If we were the size of a planet, and our eye cells were the size of a radio-telescope dish we would be able to "see" in those wavelengths. In fact, when we see images taken by radio telescopes, those have been essentially pitch-shifted up to something we can see, like the reverse of what we do when listening for bat clicks (where the pitch is downshifted to our hearing range).

The wikipedia article has a nice little diagram putting the wavelengths into perspective. https://en.wikipedia.org/wiki/Electromagnetic_spectrum

This makes me think of the dual slit experiment. Does the universe treat everything as a wave to save CPU cycles or something?

If we think of light as little balls (at our size i think that would make sense). If we were much bigger, we would think of these longer waves as balls too?

One way of thinking of it - everything is a wave until you make a measurement. Then you get a "collapse" (localization) of the wavefunction. Which leads to the question of why the wave function collapses - i.e. how does nature know we want to make a measurement.

Which leads to all sorts of crazy ideas like the simulation hypothesis. Which is not a scientific theory, because you can't falsify it, but even very educated people like Neil DeGrasse Tyson have remarked on it.

Then again RF photons just don't fit through the pupils and will get backscattered, i guess.