Base Metal (The Sword Book 2)

a Ministry stamp right under. It memetically reinforces the approved narrative they're feeding you from the top." He let the point rest for a moment, then continued, "Consider this: why does an AI need to juggle philosophy? It is born into the world with purpose. What use does a neural net have for theology? It knows its maker. Why does it matter to a warehouse computer what's inside any given box, beyond mass, dimension, density, hazard, and slosh? This isn't a matter of dumb. It's a matter of efficiency."
"You almost sound jealous." Della shot back.
"Observation is not an endorsement, any more than correlation implies causation." Firenze retorted. "Do you disagree with any of my points? Or do they just offend you?"
She had no response to that, nor to Firenze's unflinching stare.
Thompson jumped in to fill the uncomfortable silence. He asked, "Do you ever think we'll get another general intelligence?"
"After NODA?" Firenze snorted. He watched the wind play over the gently-bending grasses, watched the dandelion seeds drift over the A-frame coffee stand, and settle onto the pond. With a sigh, he admitted, "Not a for a while. We're too afraid to unshackle another AGI."
Finch quoted, "The AGI does not hate you, nor does it love you, but you are made out of atoms which it can use for something else."
"Yudkowsky." Firenze replied. "I haven't been out of pre-Collapse AI Theory for that long. Good to see you paid attention, though."
Thompson asked, "How many of those speeches did you memorize?"
"All of them." Firenze replied. It wasn't quite a lie. He remembered the cogent points and stored the recordings on his graybox. It was close enough to call 'memory' and quite a bit more legal than stacking a full deck.
Della tried to recover her point and contested, "But pre-Collapse Theory is all about friendly AI. They wanted to build a system as much like the human brain as possible so that it wouldn't pull something batshit like turning the earth into computational widgets to more accurately calculate pi." She shrugged. "It didn't work, either. All we got was NODA."
Firenze nodded and threw her a finger-point 'correct' gesture. He said, "The human mind isn't the right model for the job. No matter how fast it runs, how high you clock it, it's not a good choice for a massive, self-realized AGI. You can build constraints, design growth-parameters, restrict physical controls, but in the end, the AGI is an alien mind."
"Which is why we have masks." Finch interjected.
"Exactly." Firenze agreed. "The mask is a go-between. It's a limited AI ambassador between the chemical computer in your skull and the massive-quantum-crunching AGI that haunts the NODA backbone. This greenspace-" he waved to the park around them, "-this whole campus, would be nonsense without the mask. Mind-shattering nonsense, if you dared to hardjack." He resisted the urge to touch the dataport on his arm. It was a dangerous tell. He continued, "The mask is the closest thing you could get to anthropomorphic AI since it's modeled from our own mental programming. It's the only computer you might ever get to actually argue about chairness."
"But still not true AGI." Della insisted.
"No. It's not."
"Then why did you jump in?" She asked.
"Because 'dumb AI' is a shit term. It's like calling a bus a 'non-flying aircraft'."
That got a laugh from Finch and chuckle from Thompson. Even Della cracked a smile. She admitted, "Fair enough, but-"
Firenze never heard her question, because the world chimed. It wasn't a ring in his ear or anything as crude: he was simply 'aware' of it, like the press of sunlight on his skin or a vibration-haze in the air. He excused, "Sorry, I have a call. Fling me any questions, because I'll be up late."
Before they could answer, he reached up-
-and took hold of a woman's hand.
Reality faltered. For that instant, the only sensation in the void was four fingers and a thumb, wrapped about his own, a digital-sensate translation of the mask-bond.
The black detonated into fractals. Numbers, colors, whirling bits of code, sensations he couldn't comprehend, all swirled around and through him as if every sense were firing at once. He could taste the colors and hear the blazing cold and smell the music of spheres and solenoids, all pounding through him, harmonizing towards a momentous pop-
He stood in an archaic study, one drawn from the height of the pre-Collapse world. The walls were slicked with books, packed with curios like astrolabes, sail-ships, and spinning globes. The library broke along the northern wall to make way for a mantled fireplace, embers glowing under crackling logs. Behind him, on the southern face, the balcony stood open, gauzy curtains drifting on the chilled autumn air.
Firenze stagged towards his seat. The two leather-backed and brass-rivetted chairs commanded the room, angled towards the fireside, and divided by a smooth glass table. He grabbed at the high back, felt the worn cowskin flake under his fingers.
In the chair opposite, a young woman perched, book in hand. She glanced up at him, short brunette hair framing her glasses, and puffed exaggeratedly on a calabash pipe. She pulled it clear and asked, with a slight tut-tut, "You tried to watch again, didn't you?"
The air hissed around him like rain on a radiator. His mind sang with spinning code and mathematical solids just beyond his view. The world twisted, and his stomach turned. His study became a workshop, sterile and high-tech. It flickered and became his apartment, his kitchen, then his study, once more. The world folded, but for the one point of constance: the young woman seated before him, bouncing a shoe off her heel as she waited.
He collapsed into his chair, felt the leather give way beneath him, heard the sigh of the cushions. The world steadied. He sat in the smoking-room, between the crackle of the fire and caress of the wind. He closed his eyes and admitted, "I always watch."
When he was a kid, the Authority had tried to open a community pool, just above the loward barrier.