Rich card data
This commit is contained in:
@@ -75,6 +75,14 @@ Use another profile or a second wallet to receive a transfer. Signing submits it
|
||||
|
||||
Metadata is a **fixed four-article catalog of original starter summaries**, not a live Wikipedia API feed. The catalog is genesis-bound; expanding it currently requires a new genesis/chain. Page IDs agree across catalog and state. See [consensus, identifiers, peer RPC and limits](docs/testnet-consensus.md).
|
||||
|
||||
### Live Wikipedia card insights
|
||||
|
||||
Cards also display **optional off-chain enrichment**, separate from the minted snapshot: human-readable description, visible Wikipedia categories (up to 50), article size in bytes, other-language links (up to 500, labeled as a lower bound if truncated), Wikidata ID, current revision, and six complete UTC months of Wikimedia page views. The details show the latest monthly count, daily average, six-month total, month-over-month change, and a readership chart. Counts use English Wikipedia, all access methods, and Analytics' `user` traffic classification; classified automated traffic is excluded. Article renames can affect title-based history. Missing/incomplete API data is shown as unavailable, never invented zeroes.
|
||||
|
||||
Three **live scores** range from 0–100, rounded and capped: **Attention** = log₁₀(latest complete month's views + 1) / 6 × 100; **Depth** = log₁₀(article bytes + 1) / 6 × 100; **Reach** = log₁₀(language links + 1) / 3 × 100. Depth is a size proxy, not article quality. These scores may change; the original proof-derived Power/Defense/Knowledge, rarity, NFT ownership, and consensus do not. Live revision and retrieval time are shown explicitly, with Wikipedia/Wikimedia attribution.
|
||||
|
||||
`GET /api/wikipedia/:pageId` only enriches current catalog IDs. The node contacts fixed Wikimedia HTTPS hosts with no redirects, bounded time/response sizes, four concurrent jobs, per-article request deduplication, and a bounded cache (up to six hours; shorter for partial failures). API outages do not block mining, validation, or transfers. No live enrichment is written into blocks, NFT state, or the genesis manifest; existing chains remain compatible. Rebuild the Docker image to serve this feature from containers.
|
||||
|
||||
## Connect your own independent nodes
|
||||
|
||||
Operators must provision their own nodes, separate storage and connectivity; multiple containers on one machine are not independent security domains. Configure `WIKIDECK_PEERS` as a comma-separated list of fixed peer **origins**, for example `https://peer-a.example.org,https://peer-b.example.org` after replacing these placeholders with actual operator-controlled endpoints. Default policy requires HTTPS and public IP/DNS answers; credentials, paths, queries, fragments and redirects are rejected. At most 16 peers; no bootnodes, DHT, discovery or peer-list expansion.
|
||||
|
||||
@@ -57,4 +57,4 @@ services:
|
||||
volumes:
|
||||
testnet-a:
|
||||
testnet-b:
|
||||
testnet-c:
|
||||
testnet-c:
|
||||
|
||||
@@ -191,4 +191,4 @@ Verified on Windows x64 with CMake 3.26.4, MinGW GCC 13.1.0 and Node 24.22.0:
|
||||
- Both build scripts passed syntax checks.
|
||||
|
||||
Fast-mode allocation/runtime and Linux/macOS/MSVC builds were not run. The memory
|
||||
and initialization cost of the full dataset remains an explicit opt-in.
|
||||
and initialization cost of the full dataset remains an explicit opt-in.
|
||||
|
||||
@@ -63,9 +63,10 @@ Sync polls every **5 seconds**, checks network/chain ID, imports at most **32**
|
||||
| `GET /healthz` | Process health/network, not sync or consensus security. |
|
||||
| `GET /api/state`, `/api/chain` | Canonical blocks, mined/pending transfers, tip/work and next article. |
|
||||
| `GET /api/nfts/:id`, `/api/peers` | Canonical NFT owner/sequence/confirmations/metadata; configured peer status. |
|
||||
| `GET /api/wikipedia/:pageId` | Optional cached live enrichment for catalog IDs: categories, views, size, languages and display-only scores. Never a consensus input. |
|
||||
| `POST /api/mining/template`, `/api/blocks` | Public miner identity to template; independently validated signed block submission. |
|
||||
| `POST /api/transfer` | Signed transfer admission; 202 means pending. |
|
||||
| `GET /p2p/status`, `/p2p/blocks?offset=0`, `/p2p/mempool` | Manifest/size/tip, 32-block inventory pages, pending transactions. |
|
||||
| `POST /p2p/block`, `/p2p/transaction` | Same validation as local block/transfer submission. |
|
||||
|
||||
POSTs require JSON, enforce supplied browser origins and signatures where applicable; peers without an Origin header are not authenticated by this check. HTTP parsing is capped at 68 KiB; API/P2P rate limits are 300/minute and block/transfer POSTs additionally 30/minute. There are no fees, peer authentication or economic spam deterrents. Same-origin browser RPC trusts the node; it does not independently verify these commitments. Deploy no funds: majority-hashpower, eclipse, censorship, reorg and resource-exhaustion risks require independent audits and adversarial testing before any broader launch.
|
||||
POSTs require JSON, enforce supplied browser origins and signatures where applicable; peers without an Origin header are not authenticated by this check. HTTP parsing is capped at 68 KiB; API/P2P rate limits are 300/minute and block/transfer POSTs additionally 30/minute. There are no fees, peer authentication or economic spam deterrents. Same-origin browser RPC trusts the node; it does not independently verify these commitments. Deploy no funds: majority-hashpower, eclipse, censorship, reorg and resource-exhaustion risks require independent audits and adversarial testing before any broader launch.
|
||||
|
||||
@@ -45,4 +45,4 @@ if(WIKIDECK_RANDOMX_TESTS)
|
||||
enable_testing()
|
||||
add_test(NAME randomx-upstream-vector COMMAND wikideck-randomx --self-test)
|
||||
set_tests_properties(randomx-upstream-vector PROPERTIES TIMEOUT 120)
|
||||
endif()
|
||||
endif()
|
||||
|
||||
@@ -211,4 +211,4 @@ int main(int argc, char** argv) {
|
||||
std::cerr << "randomx: native failure\n";
|
||||
return 1;
|
||||
}
|
||||
}
|
||||
}
|
||||
|
||||
@@ -22,4 +22,4 @@ if ($Test) {
|
||||
& ctest --test-dir $BuildDirectory -C Release --output-on-failure
|
||||
if ($LASTEXITCODE -ne 0) { throw 'RandomX known-vector test failed.' }
|
||||
}
|
||||
Write-Host "Built native RandomX engine in $BuildDirectory"
|
||||
Write-Host "Built native RandomX engine in $BuildDirectory"
|
||||
|
||||
@@ -18,4 +18,4 @@ command -v git >/dev/null 2>&1 || { printf '%s\n' 'Git is required' >&2; exit 1;
|
||||
cmake -S "$root/native" -B "$build" -DCMAKE_BUILD_TYPE=Release "-DWIKIDECK_RANDOMX_TESTS=$tests"
|
||||
cmake --build "$build" --config Release --target wikideck-randomx --parallel "$jobs"
|
||||
if [ "$tests" = ON ]; then ctest --test-dir "$build" -C Release --output-on-failure; fi
|
||||
printf 'Built native RandomX engine in %s\n' "$build"
|
||||
printf 'Built native RandomX engine in %s\n' "$build"
|
||||
|
||||
@@ -56,4 +56,4 @@ try {
|
||||
if(Date.now()>deadline)throw new Error('Peer convergence timeout');
|
||||
await delay(500);
|
||||
}
|
||||
}finally{engine.close();}
|
||||
}finally{engine.close();}
|
||||
|
||||
@@ -0,0 +1,165 @@
|
||||
import { z } from 'zod';
|
||||
import { deriveStats, type PageEnrichment } from '../shared/enrichment.ts';
|
||||
|
||||
const MAX_BYTES = 1024 * 1024;
|
||||
const integer = z.number().int().min(0).max(Number.MAX_SAFE_INTEGER);
|
||||
const text = z.string().max(4096);
|
||||
const pageSchema = z.object({
|
||||
pageid: integer.positive(), ns: z.literal(0), title: text.min(1),
|
||||
missing: z.never().optional(), invalid: z.never().optional(),
|
||||
lastrevid: integer.positive(), length: integer,
|
||||
description: text.optional(),
|
||||
pageprops: z.object({ wikibase_item: z.string().regex(/^Q[1-9]\d{0,19}$/).optional() }).optional(),
|
||||
categories: z.array(z.object({ ns: z.literal(14), title: text.min(1) })).max(50).optional(),
|
||||
langlinks: z.array(z.object({ lang: z.string().min(1).max(64) })).max(500).optional(),
|
||||
});
|
||||
const metadataSchema = z.object({
|
||||
query: z.object({ pages: z.array(pageSchema).length(1) }),
|
||||
continue: z.object({ clcontinue: text.optional(), llcontinue: text.optional() }).optional(),
|
||||
});
|
||||
const viewsSchema = z.object({ items: z.array(z.object({
|
||||
timestamp: z.string().regex(/^\d{6}0100$/), views: integer,
|
||||
project: z.literal('en.wikipedia'), access: z.literal('all-access'),
|
||||
agent: z.literal('user'), granularity: z.literal('monthly'),
|
||||
})).max(6) });
|
||||
|
||||
type CacheEntry = { expires: number; value: PageEnrichment } | { expires: number; error: Error };
|
||||
type Options = { fetch?: typeof globalThis.fetch; now?: () => Date; ttlMs?: number; maxEntries?: number };
|
||||
|
||||
function monthWindow(now: Date): string[] {
|
||||
return Array.from({ length: 6 }, (_, index) =>
|
||||
new Date(Date.UTC(now.getUTCFullYear(), now.getUTCMonth() - 6 + index, 1)).toISOString().slice(0, 7));
|
||||
}
|
||||
|
||||
/** Fixed-source read-only enrichment; no consensus, persistence, URL or header inputs. */
|
||||
export class WikipediaEnrichment {
|
||||
private readonly fetcher: typeof globalThis.fetch;
|
||||
private readonly now: () => Date;
|
||||
private readonly ttlMs: number;
|
||||
private readonly maxEntries: number;
|
||||
private readonly cache = new Map<number, CacheEntry>();
|
||||
private readonly pending = new Map<number, Promise<PageEnrichment>>();
|
||||
private active = 0;
|
||||
|
||||
constructor(options: Options = {}) {
|
||||
this.fetcher = options.fetch ?? globalThis.fetch;
|
||||
this.now = options.now ?? (() => new Date());
|
||||
this.ttlMs = options.ttlMs ?? 6 * 60 * 60 * 1000;
|
||||
this.maxEntries = options.maxEntries ?? 128;
|
||||
if (!Number.isSafeInteger(this.ttlMs) || this.ttlMs <= 0 ||
|
||||
!Number.isSafeInteger(this.maxEntries) || this.maxEntries <= 0 || this.maxEntries > 128) {
|
||||
throw new RangeError('Invalid enrichment cache options');
|
||||
}
|
||||
}
|
||||
|
||||
async get(pageId: number): Promise<PageEnrichment> {
|
||||
if (!Number.isSafeInteger(pageId) || pageId <= 0) throw new RangeError('Invalid Wikipedia page ID');
|
||||
const now = this.now();
|
||||
const cached = this.cache.get(pageId);
|
||||
if (cached && cached.expires > now.getTime() &&
|
||||
(!('value' in cached) || cached.value.pageviews.latestMonth === monthWindow(now)[5])) {
|
||||
this.cache.delete(pageId);
|
||||
this.cache.set(pageId, cached);
|
||||
if ('error' in cached) throw cached.error;
|
||||
return structuredClone(cached.value);
|
||||
}
|
||||
this.cache.delete(pageId);
|
||||
const existing = this.pending.get(pageId);
|
||||
if (existing) return structuredClone(await existing);
|
||||
// Reject immediately: no unbounded waiting queue. Four active jobs also keep
|
||||
// the pending-map ceiling of eight conservative (one upstream request/job).
|
||||
if (this.pending.size >= 8 || this.active >= 4) throw new Error('Wikipedia enrichment busy');
|
||||
this.active++;
|
||||
const job = this.load(pageId, now).then(value => {
|
||||
const ttl = value.pageviews.status === 'available' ? this.ttlMs : Math.min(this.ttlMs, 5 * 60 * 1000);
|
||||
this.remember(pageId, { value, expires: this.now().getTime() + ttl });
|
||||
return value;
|
||||
}, () => {
|
||||
const error = new Error('Wikipedia page enrichment unavailable');
|
||||
this.remember(pageId, { error, expires: this.now().getTime() + 30_000 });
|
||||
throw error;
|
||||
}).finally(() => { this.active--; this.pending.delete(pageId); });
|
||||
this.pending.set(pageId, job);
|
||||
return structuredClone(await job);
|
||||
}
|
||||
|
||||
private remember(pageId: number, entry: CacheEntry): void {
|
||||
this.cache.delete(pageId);
|
||||
this.cache.set(pageId, entry);
|
||||
while (this.cache.size > this.maxEntries) this.cache.delete(this.cache.keys().next().value!);
|
||||
}
|
||||
|
||||
private async json(url: URL): Promise<unknown> {
|
||||
const signal = AbortSignal.timeout(6000);
|
||||
const response = await this.fetcher(url, { signal, redirect: 'error',
|
||||
headers: { Accept: 'application/json', 'User-Agent': 'Wikideck/0.1 (Wikipedia off-chain page enrichment)' } });
|
||||
const length = response.headers.get('content-length');
|
||||
if (!response.ok || response.redirected ||
|
||||
!/^application\/json(?:\s*;|\s*$)/i.test(response.headers.get('content-type') ?? '') ||
|
||||
(length !== null && (!/^\d+$/.test(length) || Number(length) > MAX_BYTES)) || !response.body) {
|
||||
await response.body?.cancel();
|
||||
throw new Error('Invalid upstream response');
|
||||
}
|
||||
const reader = response.body.getReader();
|
||||
const chunks: Uint8Array[] = [];
|
||||
let size = 0;
|
||||
try {
|
||||
for (;;) {
|
||||
signal.throwIfAborted();
|
||||
const { done, value } = await reader.read();
|
||||
signal.throwIfAborted();
|
||||
if (done) break;
|
||||
size += value.byteLength;
|
||||
if (size > MAX_BYTES) throw new Error('Upstream response too large');
|
||||
chunks.push(value);
|
||||
}
|
||||
return JSON.parse(new TextDecoder('utf-8', { fatal: true }).decode(Buffer.concat(chunks)));
|
||||
} finally {
|
||||
await reader.cancel();
|
||||
reader.releaseLock();
|
||||
}
|
||||
}
|
||||
|
||||
private async load(pageId: number, now: Date): Promise<PageEnrichment> {
|
||||
const url = new URL('https://en.wikipedia.org/w/api.php');
|
||||
url.search = new URLSearchParams({ action: 'query', format: 'json', formatversion: '2',
|
||||
pageids: String(pageId), prop: 'info|categories|pageprops|description|langlinks',
|
||||
clshow: '!hidden', cllimit: '50', lllimit: '500', ppprop: 'wikibase_item' }).toString();
|
||||
const metadata = metadataSchema.parse(await this.json(url));
|
||||
const page = metadata.query.pages[0];
|
||||
if (page.pageid !== pageId) throw new Error('Wikipedia page ID mismatch');
|
||||
const months = monthWindow(now);
|
||||
const latestMonth = months[5];
|
||||
let pageviews: PageEnrichment['pageviews'] = { status: 'unavailable', months: [], latestMonth,
|
||||
total: null, dailyAverage: null, trendPercent: null };
|
||||
try {
|
||||
const title = encodeURIComponent(page.title.replace(/ /g, '_'));
|
||||
const stamp = (month: string) => `${month.replace('-', '')}0100`;
|
||||
const endpoint = new URL(`https://wikimedia.org/api/rest_v1/metrics/pageviews/per-article/en.wikipedia.org/all-access/user/${title}/monthly/${stamp(months[0])}/${stamp(latestMonth)}`);
|
||||
const result = viewsSchema.parse(await this.json(endpoint));
|
||||
const byMonth = new Map<string, number>();
|
||||
for (const item of result.items) {
|
||||
const month = `${item.timestamp.slice(0, 4)}-${item.timestamp.slice(4, 6)}`;
|
||||
if (!months.includes(month) || byMonth.has(month)) throw new Error('Invalid pageview month');
|
||||
byMonth.set(month, item.views);
|
||||
}
|
||||
if (byMonth.size !== 6) throw new Error('Incomplete pageviews');
|
||||
const ordered = months.map(month => ({ month, views: byMonth.get(month)! }));
|
||||
const total = ordered.reduce((sum, item) => sum + item.views, 0);
|
||||
if (!Number.isSafeInteger(total)) throw new Error('Pageview total overflow');
|
||||
const latest = ordered[5].views, prior = ordered[4].views;
|
||||
const days = new Date(Date.UTC(now.getUTCFullYear(), now.getUTCMonth(), 0)).getUTCDate();
|
||||
pageviews = { status: 'available', months: ordered, latestMonth, total,
|
||||
dailyAverage: latest / days, trendPercent: prior > 0 ? (latest - prior) / prior * 100 : null };
|
||||
} catch { /* Metadata remains useful; unknown views must not become fake zeroes. */ }
|
||||
return { pageId, title: page.title, description: page.description ?? null,
|
||||
wikidataId: page.pageprops?.wikibase_item ?? null, revisionId: page.lastrevid, pageBytes: page.length,
|
||||
languageCount: page.langlinks?.length ?? 0, languagesTruncated: metadata.continue?.llcontinue !== undefined,
|
||||
categories: (page.categories ?? []).map(item => item.title.replace(/^Category:/, '')),
|
||||
categoriesTruncated: metadata.continue?.clcontinue !== undefined, fetchedAt: now.toISOString(), pageviews,
|
||||
stats: deriveStats(page.length, page.langlinks?.length ?? 0,
|
||||
pageviews.status === 'available' ? pageviews.months[5].views : null) };
|
||||
}
|
||||
}
|
||||
|
||||
export const wikipediaEnrichment = new WikipediaEnrichment();
|
||||
+1
-1
@@ -1,2 +1,2 @@
|
||||
// A node independently validates RandomX work and follows cumulative-work consensus.
|
||||
import './testnet/http.ts';
|
||||
import './testnet/http.ts';
|
||||
|
||||
@@ -139,4 +139,4 @@ export async function validateBlock(raw: unknown, ancestors: TestnetBlock[], par
|
||||
if (BigInt(`0x${b.powHash}`) > BigInt(`0x${b.target}`)) throw new Error('Insufficient work');
|
||||
if (await pow.hash(b.key, blob) !== b.powHash) throw new Error('Invalid RandomX proof');
|
||||
return { block: b, state: next };
|
||||
}
|
||||
}
|
||||
|
||||
+13
-1
@@ -10,6 +10,8 @@ import { CHAIN_ID, blockSchema, transferSchema } from './consensus.ts';
|
||||
import { RandomX } from './randomx.ts';
|
||||
import { Ledger } from './ledger.ts';
|
||||
import { PeerNetwork } from './peers.ts';
|
||||
import { wikipediaEnrichment } from '../enrichment.ts';
|
||||
import { STARTER_ARTICLES } from '../../shared/catalog.ts';
|
||||
|
||||
const root = fileURLToPath(new URL('../../', import.meta.url));
|
||||
const dataDir = process.env.WIKIDECK_DATA_DIR ?? resolve(root,'data');
|
||||
@@ -50,6 +52,16 @@ app.use(['/api','/p2p'],(req,res,next)=>{
|
||||
app.get('/api/state',(_req,res)=>res.json(ledger.snapshot()));
|
||||
app.get('/api/chain',(_req,res)=>res.json(ledger.snapshot()));
|
||||
app.get('/api/peers',(_req,res)=>res.json({peers:peers.status()}));
|
||||
// Optional live data is intentionally not used by genesis, block validation or NFT ownership.
|
||||
app.get('/api/wikipedia/:pageId', rateLimit({windowMs:60000,limit:60,standardHeaders:'draft-8',legacyHeaders:false}), async(req,res)=>{
|
||||
if(typeof req.params.pageId!=='string'||!/^[1-9]\d{0,9}$/.test(req.params.pageId)){res.status(400).json({error:'Invalid Wikipedia page ID'});return;}
|
||||
const pageId=Number(req.params.pageId);
|
||||
if(!STARTER_ARTICLES.some(a=>a.pageId===pageId)){
|
||||
res.status(404).json({error:'Page is not in the current consensus catalog'});return;
|
||||
}
|
||||
try {res.json(await wikipediaEnrichment.get(pageId));}
|
||||
catch {res.status(503).json({error:'Wikipedia enrichment is temporarily unavailable. NFT data is unaffected.'});}
|
||||
});
|
||||
app.get('/p2p/status',(_req,res)=>res.json({network:NETWORK,chainId:CHAIN_ID,size:ledger.size,tip:ledger.snapshot().tip}));
|
||||
app.get('/p2p/blocks',(req,res)=>{
|
||||
const offset=Number(req.query.offset??0);
|
||||
@@ -103,4 +115,4 @@ for(const signal of ['SIGTERM','SIGINT']as const)process.once(signal,()=>{
|
||||
const timeout=setTimeout(()=>process.exit(1),25000);timeout.unref();
|
||||
server.close(async()=>{engine.close();await lock.close();await unlink(lockPath).catch(()=>{});clearTimeout(timeout);process.exit(0);});
|
||||
});
|
||||
server.on('error',async()=>{peers.close();engine.close();await lock.close();await unlink(lockPath).catch(()=>{});process.exit(1);});
|
||||
server.on('error',async()=>{peers.close();engine.close();await lock.close();await unlink(lockPath).catch(()=>{});process.exit(1);});
|
||||
|
||||
@@ -210,4 +210,4 @@ export class Ledger {
|
||||
}
|
||||
} finally { await unlink(temp).catch(() => undefined); }
|
||||
}
|
||||
}
|
||||
}
|
||||
|
||||
@@ -90,4 +90,4 @@ try {
|
||||
}
|
||||
}
|
||||
} catch(e) { if(!stopping) { console.error((e as Error).message); process.exitCode=1; } }
|
||||
finally { engine.close(); process.off('SIGINT',stop);process.off('SIGTERM',stop); }
|
||||
finally { engine.close(); process.off('SIGINT',stop);process.off('SIGTERM',stop); }
|
||||
|
||||
@@ -192,4 +192,4 @@ export class PeerNetwork {
|
||||
});
|
||||
} finally { clearTimeout(timeout); this.active.delete(controller); }
|
||||
}
|
||||
}
|
||||
}
|
||||
|
||||
@@ -205,4 +205,4 @@ export class RandomX implements PowEngine {
|
||||
this.shutdownTimer.unref();
|
||||
}
|
||||
}
|
||||
}
|
||||
}
|
||||
|
||||
@@ -0,0 +1,36 @@
|
||||
/** Live, off-chain Wikipedia data. Text fields are plain strings, never HTML. */
|
||||
export interface PageEnrichment {
|
||||
pageId: number;
|
||||
title: string;
|
||||
description: string | null;
|
||||
wikidataId: string | null;
|
||||
revisionId: number;
|
||||
pageBytes: number;
|
||||
/** Number of returned interlanguage links (a lower bound when truncated). */
|
||||
languageCount: number;
|
||||
languagesTruncated: boolean;
|
||||
categories: string[];
|
||||
categoriesTruncated: boolean;
|
||||
fetchedAt: string;
|
||||
pageviews: {
|
||||
status: 'available' | 'unavailable';
|
||||
/** Six completed UTC months, oldest first, identified as YYYY-MM. */
|
||||
months: { month: string; views: number }[];
|
||||
latestMonth: string;
|
||||
total: number | null;
|
||||
/** Latest completed month's views divided by its actual UTC day count. */
|
||||
dailyAverage: number | null;
|
||||
trendPercent: number | null;
|
||||
};
|
||||
stats: { attention: number | null; depth: number; reach: number };
|
||||
}
|
||||
|
||||
/** Attention uses the latest completed month's views, not the six-month total. */
|
||||
export function deriveStats(pageBytes: number, languageCount: number, monthlyViews: number | null): PageEnrichment['stats'] {
|
||||
const score = (value: number, scale: number) => {
|
||||
if (!Number.isFinite(value) || value < 0) throw new RangeError('Invalid enrichment score input');
|
||||
return Math.min(100, Math.round(Math.log10(value + 1) / scale * 100));
|
||||
};
|
||||
return { attention: monthlyViews === null ? null : score(monthlyViews, 6),
|
||||
depth: score(pageBytes, 6), reach: score(languageCount, 3) };
|
||||
}
|
||||
+1
-1
@@ -46,4 +46,4 @@ export function testnetOwnerOf(state: Pick<TestnetSnapshot, 'blocks' | 'transfer
|
||||
let owner = state.blocks.find(b => b.tokenId === hash)?.owner;
|
||||
for (const t of state.transfers) if (t.cardHash === hash) owner = t.to;
|
||||
return owner;
|
||||
}
|
||||
}
|
||||
|
||||
@@ -142,4 +142,4 @@ export async function decryptWallet(backup: unknown, password: string): Promise<
|
||||
} catch {
|
||||
throw new Error('Cannot unlock backup: incorrect password, damaged backup, or mismatched wallet keys.');
|
||||
} finally { plaintext?.fill(0); }
|
||||
}
|
||||
}
|
||||
|
||||
+7
-3
@@ -2,6 +2,8 @@ import { ArrowUpRight, BookOpen, Globe2, Shield, Sparkles, Swords } from 'lucide
|
||||
import { rarity, stats, type Article } from '../shared/protocol.ts';
|
||||
import type { TestnetBlock } from '../shared/testnet.ts';
|
||||
import { STARTER_ARTICLES } from '../shared/catalog.ts';
|
||||
import { useEnrichment } from './useEnrichment.ts';
|
||||
import { CardInsights, EnrichmentScores } from './CardInsights.tsx';
|
||||
|
||||
// hash is the RandomX proof hash for game stats; block.tokenId is NFT identity.
|
||||
export type CardData = { article: Article; hash: string; block?: TestnetBlock };
|
||||
@@ -19,15 +21,17 @@ export function Card({ card, onClick }: { card: CardData; onClick?: () => void }
|
||||
const { article, hash } = card;
|
||||
const tier = rarity(hash);
|
||||
const s = stats(hash);
|
||||
return <button className={`wiki-card ${tier.toLowerCase()}`} onClick={onClick} tabIndex={onClick ? 0 : -1} aria-label={`View ${article.title} ${tier} card`}>
|
||||
const {data:enrichment,error}=useEnrichment(article.pageId);
|
||||
return <><button className={`wiki-card ${tier.toLowerCase()}`} onClick={onClick} tabIndex={onClick ? 0 : -1} aria-label={`View ${article.title} ${tier} card`}>
|
||||
<div className="card-top"><span><Sparkles size={11} /> {tier}</span><span>W / {String(article.pageId).slice(-4)}</span></div>
|
||||
<div className={`card-art art-${article.category.toLowerCase()}`}>
|
||||
{article.image ? <img src={article.image} alt={article.title} loading="lazy" referrerPolicy="no-referrer" onError={e => { e.currentTarget.style.display = 'none'; }} /> : <Globe2 size={64} />}
|
||||
<span className="card-category">{article.category}</span>
|
||||
</div>
|
||||
<div className="card-content"><div className="card-title"><h3>{article.title}</h3><ArrowUpRight size={16} /></div><p>{article.extract}</p>
|
||||
<div className="card-stats"><span><Swords size={12} />{s.power}</span><span><Shield size={12} />{s.defense}</span><span><BookOpen size={12} />{s.knowledge}</span></div>
|
||||
<div className="card-stats" aria-label="Immutable proof-derived game stats"><span title="Power · proof-derived"><Swords size={12} />{s.power}</span><span title="Defense · proof-derived"><Shield size={12} />{s.defense}</span><span title="Knowledge · proof-derived"><BookOpen size={12} />{s.knowledge}</span></div>
|
||||
<div className="card-enrichment">{enrichment?<><div className="card-readership"><span>LIVE WIKIPEDIA</span><span>{enrichment.pageviews.status==='available'?`${new Intl.NumberFormat('en',{notation:'compact',maximumFractionDigits:1}).format(enrichment.pageviews.months.at(-1)!.views)} views / month`:'Views unavailable'}</span></div><EnrichmentScores data={enrichment}/></>:<span className="card-enrichment-status">{error?'Live metadata unavailable':'Loading article insights…'}</span>}</div>
|
||||
</div>
|
||||
<div className="card-bottom"><span>WIKIPEDIA · #{article.pageId}</span><span>{card.block ? `BLOCK ${card.block.height}` : 'PREVIEW'}</span></div>
|
||||
</button>;
|
||||
</button>{!onClick&&<CardInsights pageId={article.pageId}/>}</>;
|
||||
}
|
||||
|
||||
@@ -0,0 +1,44 @@
|
||||
import { BookOpen, Eye, Globe2, Tags, TrendingUp } from 'lucide-react';
|
||||
import { useEnrichment } from './useEnrichment.ts';
|
||||
import type { PageEnrichment } from '../shared/enrichment.ts';
|
||||
import './enrichment.css';
|
||||
|
||||
const compact=new Intl.NumberFormat('en',{notation:'compact',maximumFractionDigits:1});
|
||||
const full=new Intl.NumberFormat('en',{maximumFractionDigits:0});
|
||||
const month=(value:string)=>new Date(`${value}-01T00:00:00Z`).toLocaleDateString('en',{month:'short',year:'numeric',timeZone:'UTC'});
|
||||
const score=(value:number|null)=>value===null?'—':`${value}`;
|
||||
export function EnrichmentScores({data}: {data:PageEnrichment}){
|
||||
return <div className="enrichment-scores" aria-label="Live Wikipedia-derived scores out of 100">
|
||||
<span title="Attention: logarithmic score from the latest complete month's human page views"><Eye size={12}/><b>{score(data.stats.attention)}</b><small>Attention</small></span>
|
||||
<span title="Depth: logarithmic score from article bytes, not a quality judgment"><BookOpen size={12}/><b>{score(data.stats.depth)}</b><small>Depth</small></span>
|
||||
<span title="Reach: logarithmic score from interlanguage links"><Globe2 size={12}/><b>{score(data.stats.reach)}</b><small>Reach</small></span>
|
||||
</div>;
|
||||
}
|
||||
export function CardInsights({pageId}:{pageId:number}){
|
||||
const {data,error,retry}=useEnrichment(pageId);
|
||||
if(!data)return <section className="article-insights" aria-label="Wikipedia insights"><h3>Wikipedia insights</h3><p role="status">{error?'Live metadata is unavailable. The NFT and its original game stats are unchanged.':'Loading Wikipedia metadata and monthly readership…'}</p>{error&&<button className="text-button" onClick={retry}>Retry live metadata</button>}</section>;
|
||||
const views=data.pageviews;
|
||||
const latest=views.months.at(-1);
|
||||
const maximum=Math.max(1,...views.months.map(m=>m.views));
|
||||
const trend=views.trendPercent;
|
||||
return <section className="article-insights" aria-label="Wikipedia insights">
|
||||
<div className="insights-heading"><h3>Wikipedia insights</h3><span>LIVE · OFF-CHAIN</span></div>
|
||||
{data.description&&<p>{data.description}</p>}
|
||||
<EnrichmentScores data={data}/>
|
||||
<dl className="article-fields">
|
||||
<div><dt><Eye size={12}/> Monthly views</dt><dd>{latest?full.format(latest.views):'Unavailable'}<small>{month(views.latestMonth)} · human traffic</small></dd></div>
|
||||
<div><dt><TrendingUp size={12}/> Monthly change</dt><dd>{trend===null?'Unavailable':`${trend>=0?'+':''}${trend.toFixed(1)}%`}<small>vs. preceding month</small></dd></div>
|
||||
<div><dt>6-month views</dt><dd>{views.total===null?'Unavailable':full.format(views.total)}</dd></div>
|
||||
<div><dt>Daily average</dt><dd>{views.dailyAverage===null?'Unavailable':full.format(views.dailyAverage)}<small>latest complete month</small></dd></div>
|
||||
<div><dt>Article size</dt><dd>{full.format(data.pageBytes)} bytes</dd></div>
|
||||
<div><dt>Languages</dt><dd>{data.languagesTruncated?'≥ ':''}{full.format(data.languageCount)}<small>other-language links</small></dd></div>
|
||||
<div><dt>Live revision</dt><dd><a href={`https://en.wikipedia.org/w/index.php?oldid=${data.revisionId}`} target="_blank" rel="noreferrer">{data.revisionId}</a></dd></div>
|
||||
<div><dt>Wikidata</dt><dd>{data.wikidataId?<a href={`https://www.wikidata.org/wiki/${data.wikidataId}`} target="_blank" rel="noreferrer">{data.wikidataId}</a>:'Not linked'}</dd></div>
|
||||
</dl>
|
||||
{views.status==='available'?<figure className="pageview-history"><figcaption>Monthly readership · six complete UTC months</figcaption><div className="pageview-bars">{views.months.map(item=><div key={item.month} title={`${month(item.month)}: ${full.format(item.views)} views`}><span>{compact.format(item.views)}</span><div className="pageview-track"><i style={{height:`${Math.max(1,item.views/maximum*100)}%`}}/></div><small>{month(item.month)}</small></div>)}</div><table className="visually-hidden"><caption>Monthly human page views</caption><tbody>{views.months.map(item=><tr key={item.month}><th>{month(item.month)}</th><td>{item.views}</td></tr>)}</tbody></table></figure>:<p className="insights-unavailable">Page views are unavailable or incomplete. Unknown counts are not treated as zero.</p>}
|
||||
<h4 className="insights-category-heading"><Tags size={13}/> Wikipedia categories{data.categoriesTruncated?' · first 50':''}</h4>
|
||||
<div className="article-categories">{data.categories.length?data.categories.map(category=><a key={category} href={`https://en.wikipedia.org/wiki/${encodeURIComponent(`Category:${category.replace(/ /g,'_')}`)}`} target="_blank" rel="noreferrer">{category}</a>):<span>No visible categories returned</span>}</div>
|
||||
<details className="insights-method"><summary>How are these scores calculated?</summary><p>Scores range from 0–100, capped and rounded: Attention = log₁₀(monthly views + 1) ÷ 6 × 100; Depth = log₁₀(article bytes + 1) ÷ 6 × 100; Reach = log₁₀(language links + 1) ÷ 3 × 100. Depth measures size, not accuracy or quality. Scores can change and do not affect proof-derived rarity, NFT ownership, or consensus.</p></details>
|
||||
<p className="insights-attribution">Retrieved {new Date(data.fetchedAt).toLocaleString()}. Live revision may differ from the minted snapshot. Sources: <a href={`https://en.wikipedia.org/?curid=${pageId}`} target="_blank" rel="noreferrer">Wikipedia contributors</a> (CC BY-SA 4.0) and Wikimedia Analytics (CC0). Views use all-access, user traffic and exclude classified bots. Article moves may affect title-based view history.</p>
|
||||
</section>;
|
||||
}
|
||||
+1
-1
@@ -122,4 +122,4 @@ export function WalletPanel({ wallet, onRestore, onNotice }: Props) {
|
||||
</section>;
|
||||
}
|
||||
|
||||
export default WalletPanel;
|
||||
export default WalletPanel;
|
||||
|
||||
@@ -0,0 +1,2 @@
|
||||
.card-modal{grid-template-areas:"card detail" "insights insights"}.card-modal .detail-card{display:contents}.card-modal .detail-card>.wiki-card{grid-area:card;align-self:start}.card-modal .detail-copy{grid-area:detail;min-width:0}.card-modal .article-insights{grid-area:insights;margin-top:0}.card-modal .article-fields{grid-template-columns:repeat(4,minmax(0,1fr))}@media(max-width:560px){.card-modal .detail-card>.wiki-card{margin-bottom:24px}.card-modal .article-insights{margin:24px 0}.card-modal .article-fields{grid-template-columns:repeat(2,minmax(0,1fr))}}
|
||||
.card-enrichment{border-top:1px solid var(--theme-border);margin-top:12px;padding-top:10px;min-height:68px}.card-readership{display:flex;justify-content:space-between;gap:6px;font-size:8px;color:var(--ctp-subtext0)}.card-readership>span:first-child{font-size:7px;letter-spacing:.7px;color:var(--ctp-teal)}.enrichment-scores{display:grid;grid-template-columns:repeat(3,minmax(0,1fr));gap:8px;margin-top:10px}.enrichment-scores>span{display:grid;grid-template-columns:auto auto;justify-content:center;align-items:center;gap:3px 5px;font-size:11px;color:var(--ctp-lavender)}.enrichment-scores b{font-weight:600;color:var(--ctp-text)}.enrichment-scores small{grid-column:1/-1;text-align:center;font-size:8px;color:var(--ctp-subtext0)}.card-enrichment-status{color:var(--ctp-subtext0);font-size:9px}.hero-card .card-enrichment{display:none}.article-insights{margin-top:24px;padding:20px;background:var(--ctp-mantle);border:1px solid var(--ctp-surface1);border-radius:10px;text-align:left;pointer-events:auto}.article-insights h3{font-size:18px;letter-spacing:-.3px;font-weight:550;color:var(--ctp-text)}.article-insights p{font-size:11px;line-height:1.85;color:var(--ctp-subtext0);margin-top:12px}.insights-heading{display:flex;flex-wrap:wrap;align-items:center;justify-content:space-between;gap:9px}.insights-heading>span{font-size:8px;letter-spacing:1px;color:var(--ctp-teal)}.article-insights .enrichment-scores{padding:16px 0;border-bottom:1px solid var(--theme-border)}.article-insights .enrichment-scores b{font-size:21px}.article-insights .enrichment-scores small{font-size:10px}.article-fields{display:grid;grid-template-columns:repeat(2,minmax(0,1fr));gap:18px 14px;margin:20px 0}.article-fields dt{display:flex;align-items:center;gap:5px;font-size:10px;color:var(--ctp-subtext0)}.article-fields dd{margin:6px 0 0;font-size:14px;color:var(--ctp-text);overflow-wrap:anywhere}.article-fields dd small{display:block;font-size:9px;color:var(--ctp-subtext0);margin-top:4px}.article-insights a{color:var(--ctp-blue);text-decoration:underline;text-underline-offset:3px}.pageview-history{margin:22px 0}.pageview-history figcaption{font-size:10px;color:var(--ctp-subtext1);margin-bottom:14px}.pageview-bars{display:grid;grid-template-columns:repeat(6,minmax(0,1fr));gap:7px}.pageview-bars>div{text-align:center;min-width:0}.pageview-bars span{font-size:8px;color:var(--ctp-subtext1)}.pageview-track{height:80px;display:flex;align-items:flex-end;margin-top:5px;background:var(--ctp-base);border-radius:4px;overflow:hidden}.pageview-track i{display:block;width:100%;background:var(--ctp-mauve);border-radius:3px 3px 0 0}.pageview-bars small{display:block;margin-top:7px;font-size:8px;color:var(--ctp-subtext0);line-height:1.6}.insights-category-heading{display:flex;align-items:center;gap:6px;color:var(--ctp-subtext1);font-size:11px;margin:20px 0 12px}.article-categories{display:flex;flex-wrap:wrap;gap:6px;max-height:170px;overflow:auto}.article-categories a,.article-categories>span{font-size:9px;line-height:1.6;border-radius:4px;background:var(--ctp-surface0);padding:4px 7px;text-decoration:none;color:var(--ctp-subtext1);overflow-wrap:anywhere}.article-categories a:hover{color:var(--ctp-mauve)}.insights-method{margin-top:20px;color:var(--ctp-lavender);font-size:10px;line-height:1.8}.insights-method summary{cursor:pointer}.article-insights p.insights-attribution{font-size:9px;overflow-wrap:anywhere}.visually-hidden{position:absolute!important;width:1px;height:1px;padding:0;margin:-1px;overflow:hidden;clip:rect(0,0,0,0);white-space:nowrap;border:0}.card-modal .detail-card{align-self:start}.card-modal .detail-copy{align-self:start}.card-modal .detail-card .wiki-card{pointer-events:none}@media(max-width:560px){.card-readership{flex-direction:column;font-size:8px;gap:4px}.enrichment-scores{gap:4px}.enrichment-scores>span{font-size:10px}.enrichment-scores small{font-size:7px}.article-insights{padding:16px}.card-modal .detail-card{max-width:none}.card-modal .detail-card>.wiki-card{max-width:230px;margin-inline:auto}}
|
||||
+1
-1
@@ -59,4 +59,4 @@
|
||||
.native-mining .miner-footnote { font-size: 11px; }
|
||||
.wallet-recovery fieldset { padding: 14px; }
|
||||
.wallet-recovery h2 { font-size: 20px; }
|
||||
}
|
||||
}
|
||||
|
||||
@@ -0,0 +1,28 @@
|
||||
import { useEffect, useState } from 'react';
|
||||
import type { PageEnrichment } from '../shared/enrichment.ts';
|
||||
|
||||
// One request per article shared by hero, collection and detail views. No wallet data is sent.
|
||||
const cache = new Map<number, { expires: number; job: Promise<PageEnrichment> }>();
|
||||
function load(pageId: number): Promise<PageEnrichment> {
|
||||
const existing=cache.get(pageId);
|
||||
if(existing && existing.expires>Date.now())return existing.job;
|
||||
const job=(async()=>{
|
||||
const response=await fetch(`/api/wikipedia/${pageId}`,{signal:AbortSignal.timeout(15000),cache:'no-store'});
|
||||
if(!response.ok)throw new Error('Live Wikipedia data unavailable');
|
||||
return await response.json() as PageEnrichment;
|
||||
})();
|
||||
cache.set(pageId,{expires:Date.now()+5*60*1000,job});
|
||||
if(cache.size>128)cache.delete(cache.keys().next().value!);
|
||||
return job;
|
||||
}
|
||||
export function useEnrichment(pageId: number) {
|
||||
const [data,setData]=useState<PageEnrichment>();
|
||||
const [error,setError]=useState(false);
|
||||
const [attempt,setAttempt]=useState(0);
|
||||
useEffect(()=>{
|
||||
let active=true;setData(undefined);setError(false);
|
||||
load(pageId).then(value=>{if(active)setData(value);},()=>{if(active)setError(true);});
|
||||
return()=>{active=false;};
|
||||
},[pageId,attempt]);
|
||||
return {data,error,retry:()=>{cache.delete(pageId);setAttempt(n=>n+1);}};
|
||||
}
|
||||
@@ -181,6 +181,9 @@ test('native RandomX HTTP: three blocks, mined transfer, peer gossip/sync and du
|
||||
assert.deepEqual(await health.json(), { status: 'ok', network: NETWORK });
|
||||
assert.equal(health.headers.get('cache-control'), 'no-store');
|
||||
assert.equal(health.headers.has('ratelimit'), false);
|
||||
await (await request(first.base, '/api/wikipedia/0', 400)).text();
|
||||
await (await request(first.base, '/api/wikipedia/not-a-page', 400)).text();
|
||||
await (await request(first.base, '/api/wikipedia/9999999999', 404)).text();
|
||||
const initial = await snapshot(first.base);
|
||||
assert.equal(initial.network, NETWORK);
|
||||
assert.equal(initial.chainId, CHAIN_ID);
|
||||
|
||||
@@ -0,0 +1,305 @@
|
||||
import test from 'node:test';
|
||||
import assert from 'node:assert/strict';
|
||||
import { WikipediaEnrichment } from '../server/enrichment.ts';
|
||||
import { deriveStats } from '../shared/enrichment.ts';
|
||||
|
||||
const JANUARY = new Date('2025-01-15T23:59:59Z');
|
||||
const MONTHS = ['2024-07', '2024-08', '2024-09', '2024-10', '2024-11', '2024-12'];
|
||||
function metadata(pageId = 736, title = 'Albert Einstein') {
|
||||
return { query: { pages: [{ pageid: pageId, ns: 0, title, lastrevid: 1234, length: 100_000,
|
||||
description: '<script>alert("plain text")</script>', pageprops: { wikibase_item: 'Q937' },
|
||||
categories: [{ ns: 14, title: 'Category:Physicists' }], langlinks: [{ lang: 'de', title: 'Albert Einstein' }] }] },
|
||||
continue: { clcontinue: '736|next', llcontinue: '736|next', continue: '||' } };
|
||||
}
|
||||
function views(months = MONTHS, counts = [10, 20, 30, 40, 50, 100]) {
|
||||
return { items: months.map((month, index) => ({ timestamp: `${month.replace('-', '')}0100`,
|
||||
views: counts[index], project: 'en.wikipedia', access: 'all-access', agent: 'user', granularity: 'monthly' })) };
|
||||
}
|
||||
function json(value: unknown) { return new Response(JSON.stringify(value), { headers: { 'Content-Type': 'application/json; charset=utf-8' } }); }
|
||||
function mockFetch(handler: (url: URL, init: RequestInit) => Response | Promise<Response>) {
|
||||
const calls: { url: URL; init: RequestInit }[] = [];
|
||||
const fetcher: typeof fetch = async (input, init) => {
|
||||
const url = new URL(String(input));
|
||||
calls.push({ url, init: init! });
|
||||
return handler(url, init!);
|
||||
};
|
||||
return { fetcher, calls };
|
||||
}
|
||||
function fixture(options: { now?: () => Date; ttlMs?: number; maxEntries?: number } = {}) {
|
||||
const mock = mockFetch(url => json(url.hostname === 'en.wikipedia.org'
|
||||
? metadata(Number(url.searchParams.get('pageids'))) : views()));
|
||||
return { ...mock, service: new WikipediaEnrichment({ fetch: mock.fetcher, now: () => JANUARY, ...options }) };
|
||||
}
|
||||
|
||||
test('fixed HTTPS sources, encoded title, hidden categories, lower bounds and plain strings', async () => {
|
||||
const title = 'A / B? #é%';
|
||||
const mock = mockFetch(url => json(url.hostname === 'en.wikipedia.org'
|
||||
? metadata(736, title) : { items: views().items.reverse() }));
|
||||
const service = new WikipediaEnrichment({ fetch: mock.fetcher, now: () => JANUARY });
|
||||
const result = await service.get(736);
|
||||
assert.equal(mock.calls.length, 2);
|
||||
const query = mock.calls[0].url;
|
||||
assert.equal(query.origin, 'https://en.wikipedia.org');
|
||||
assert.equal(query.pathname, '/w/api.php');
|
||||
assert.equal(query.searchParams.get('pageids'), '736');
|
||||
assert.equal(query.searchParams.get('formatversion'), '2');
|
||||
assert.equal(query.searchParams.get('prop'), 'info|categories|pageprops|description|langlinks');
|
||||
assert.equal(query.searchParams.get('clshow'), '!hidden');
|
||||
assert.equal(query.searchParams.get('cllimit'), '50');
|
||||
assert.equal(query.searchParams.get('lllimit'), '500');
|
||||
const rest = mock.calls[1].url;
|
||||
assert.equal(rest.origin, 'https://wikimedia.org');
|
||||
assert.equal(rest.pathname, `/api/rest_v1/metrics/pageviews/per-article/en.wikipedia.org/all-access/user/${encodeURIComponent(title.replace(/ /g, '_'))}/monthly/2024070100/2024120100`);
|
||||
for (const { init } of mock.calls) {
|
||||
assert.equal(init.redirect, 'error');
|
||||
assert.ok(init.signal instanceof AbortSignal);
|
||||
assert.equal(new Headers(init.headers).get('accept'), 'application/json');
|
||||
assert.match(new Headers(init.headers).get('user-agent')!, /Wikideck/);
|
||||
assert.equal(new Headers(init.headers).has('origin'), false); // Server-side requests need no CORS workaround.
|
||||
}
|
||||
assert.equal(result.title, title);
|
||||
assert.equal(result.description, '<script>alert("plain text")</script>');
|
||||
assert.equal(result.wikidataId, 'Q937');
|
||||
assert.equal(result.revisionId, 1234);
|
||||
assert.equal(result.pageBytes, 100_000);
|
||||
assert.deepEqual(result.categories, ['Physicists']);
|
||||
assert.equal(result.categoriesTruncated, true);
|
||||
assert.equal(result.languageCount, 1);
|
||||
assert.equal(result.languagesTruncated, true);
|
||||
assert.equal(result.fetchedAt, JANUARY.toISOString());
|
||||
assert.deepEqual(result.pageviews.months.map(item => item.month), MONTHS);
|
||||
assert.equal(result.pageviews.total, 250);
|
||||
assert.equal(result.pageviews.dailyAverage, 100 / 31);
|
||||
assert.equal(result.pageviews.trendPercent, 100);
|
||||
assert.deepEqual(result.stats, deriveStats(100_000, 1, 100));
|
||||
});
|
||||
|
||||
test('UTC completed-month windows cross December/January and use leap February day counts', async () => {
|
||||
for (const [date, months, days] of [
|
||||
['2024-03-01T00:01:00Z', ['2023-09', '2023-10', '2023-11', '2023-12', '2024-01', '2024-02'], 29],
|
||||
['2025-03-01T00:00:00Z', ['2024-09', '2024-10', '2024-11', '2024-12', '2025-01', '2025-02'], 28],
|
||||
['2025-01-01T00:01:00+01:00', ['2024-06', '2024-07', '2024-08', '2024-09', '2024-10', '2024-11'], 30],
|
||||
] as const) {
|
||||
const mock = mockFetch(url => json(url.hostname === 'en.wikipedia.org' ? metadata() : views([...months])));
|
||||
const result = await new WikipediaEnrichment({ fetch: mock.fetcher, now: () => new Date(date) }).get(736);
|
||||
assert.deepEqual(result.pageviews.months.map(item => item.month), months);
|
||||
assert.equal(result.pageviews.dailyAverage, 100 / days);
|
||||
}
|
||||
});
|
||||
|
||||
test('missing optional metadata has explicit defaults, not invented descriptions or counts', async () => {
|
||||
const mock = mockFetch(url => json(url.hostname === 'en.wikipedia.org'
|
||||
? { query: { pages: [{ pageid: 736, ns: 0, title: 'A', lastrevid: 1, length: 0 }] } } : views()));
|
||||
const result = await new WikipediaEnrichment({ fetch: mock.fetcher, now: () => JANUARY }).get(736);
|
||||
assert.equal(result.description, null);
|
||||
assert.equal(result.wikidataId, null);
|
||||
assert.equal(result.languageCount, 0);
|
||||
assert.equal(result.languagesTruncated, false);
|
||||
assert.equal(result.categoriesTruncated, false);
|
||||
assert.deepEqual(result.categories, []);
|
||||
});
|
||||
|
||||
test('reject invalid IDs before fetching and malformed, missing, cross-page or non-article metadata', async () => {
|
||||
const { service, calls } = fixture();
|
||||
for (const id of [0, -1, 1.2, NaN, Infinity, Number.MAX_SAFE_INTEGER + 1, '736' as unknown as number]) {
|
||||
await assert.rejects(service.get(id), /Invalid Wikipedia page ID/);
|
||||
}
|
||||
assert.equal(calls.length, 0);
|
||||
const invalid = [
|
||||
{ pageid: 737 }, { ns: 14 }, { missing: true }, { invalid: true }, { lastrevid: -1 },
|
||||
{ length: Number.MAX_SAFE_INTEGER + 1 }, { title: 'a'.repeat(4097) },
|
||||
{ categories: Array.from({ length: 51 }, () => ({ ns: 14, title: 'Category:A' })) },
|
||||
{ langlinks: Array.from({ length: 501 }, () => ({ lang: 'en' })) },
|
||||
{ pageprops: { wikibase_item: 'https://evil.example' } },
|
||||
];
|
||||
for (const patch of invalid) {
|
||||
const raw = metadata();
|
||||
Object.assign(raw.query.pages[0], patch);
|
||||
const mock = mockFetch(() => json(raw));
|
||||
await assert.rejects(new WikipediaEnrichment({ fetch: mock.fetcher }).get(736), /unavailable/);
|
||||
assert.equal(mock.calls.length, 1);
|
||||
}
|
||||
for (const raw of [{}, { query: { pages: [] } }, { query: { pages: [metadata().query.pages[0], metadata().query.pages[0]] } }]) {
|
||||
const mock = mockFetch(() => json(raw));
|
||||
await assert.rejects(new WikipediaEnrichment({ fetch: mock.fetcher }).get(736));
|
||||
}
|
||||
});
|
||||
|
||||
test('pageviews failure, missing months, duplicates and invalid values retain metadata as unavailable', async () => {
|
||||
const failures: (() => Response)[] = [
|
||||
() => new Response('', { status: 503 }), () => json({ items: [] }),
|
||||
() => json({ items: views().items.slice(1) }),
|
||||
() => json({ items: [...views().items.slice(0, 5), views().items[0]] }),
|
||||
() => json(views(['2024-06', ...MONTHS.slice(1)])),
|
||||
...[-1, 0.5, Number.MAX_SAFE_INTEGER + 1].map(value => () => json(views(MONTHS, [value, 0, 0, 0, 0, 0]))),
|
||||
() => json(views(MONTHS, Array(6).fill(Number.MAX_SAFE_INTEGER))),
|
||||
() => json({ items: views().items.map(item => ({ ...item, agent: 'spider' })) }),
|
||||
() => json({ items: views().items.map(item => ({ ...item, timestamp: '2024120200' })) }),
|
||||
() => { throw new Error('Network failure'); },
|
||||
];
|
||||
for (const failure of failures) {
|
||||
const mock = mockFetch(url => url.hostname === 'en.wikipedia.org' ? json(metadata()) : failure());
|
||||
const result = await new WikipediaEnrichment({ fetch: mock.fetcher, now: () => JANUARY }).get(736);
|
||||
assert.equal(result.title, 'Albert Einstein');
|
||||
assert.deepEqual(result.pageviews, { status: 'unavailable', months: [], latestMonth: '2024-12',
|
||||
total: null, dailyAverage: null, trendPercent: null });
|
||||
assert.equal(result.stats.attention, null);
|
||||
assert.ok(result.stats.depth > 0);
|
||||
}
|
||||
});
|
||||
|
||||
test('six real zero months remain available; zero prior month has no trend', async () => {
|
||||
const mock = mockFetch(url => json(url.hostname === 'en.wikipedia.org' ? metadata() : views(MONTHS, Array(6).fill(0))));
|
||||
const result = await new WikipediaEnrichment({ fetch: mock.fetcher, now: () => JANUARY }).get(736);
|
||||
assert.equal(result.pageviews.status, 'available');
|
||||
assert.equal(result.pageviews.total, 0);
|
||||
assert.equal(result.pageviews.dailyAverage, 0);
|
||||
assert.equal(result.pageviews.trendPercent, null);
|
||||
assert.equal(result.stats.attention, 0);
|
||||
});
|
||||
|
||||
test('logarithmic scores are rounded, capped, nullable and reject invalid inputs', () => {
|
||||
assert.deepEqual(deriveStats(0, 0, null), { attention: null, depth: 0, reach: 0 });
|
||||
assert.deepEqual(deriveStats(999, 9, 99), { attention: 33, depth: 50, reach: 33 });
|
||||
assert.deepEqual(deriveStats(Number.MAX_SAFE_INTEGER, 1e9, 1e15), { attention: 100, depth: 100, reach: 100 });
|
||||
for (const value of [-1, NaN, Infinity]) {
|
||||
assert.throws(() => deriveStats(value, 0, 0), RangeError);
|
||||
assert.throws(() => deriveStats(0, value, 0), RangeError);
|
||||
assert.throws(() => deriveStats(0, 0, value), RangeError);
|
||||
}
|
||||
});
|
||||
|
||||
test('six-hour TTL, deduplication and mutation isolation', async () => {
|
||||
let now = JANUARY.getTime();
|
||||
const { service, calls } = fixture({ now: () => new Date(now) });
|
||||
const [first, second] = await Promise.all([service.get(736), service.get(736)]);
|
||||
assert.equal(calls.length, 2);
|
||||
first.categories.push('not cached');
|
||||
second.pageviews.months[0].views = 999;
|
||||
assert.deepEqual((await service.get(736)).categories, ['Physicists']);
|
||||
assert.equal((await service.get(736)).pageviews.months[0].views, 10);
|
||||
now += 6 * 60 * 60 * 1000 - 1;
|
||||
await service.get(736);
|
||||
assert.equal(calls.length, 2);
|
||||
now++;
|
||||
await service.get(736);
|
||||
assert.equal(calls.length, 4);
|
||||
});
|
||||
|
||||
test('LRU bound and configured TTL are enforced', async () => {
|
||||
let now = JANUARY.getTime();
|
||||
const { service, calls } = fixture({ maxEntries: 2, ttlMs: 100, now: () => new Date(now) });
|
||||
await service.get(1); await service.get(2); await service.get(1); await service.get(3);
|
||||
assert.equal(calls.length, 6);
|
||||
await service.get(1);
|
||||
assert.equal(calls.length, 6);
|
||||
await service.get(2);
|
||||
assert.equal(calls.length, 8);
|
||||
now += 100;
|
||||
await service.get(2);
|
||||
assert.equal(calls.length, 10);
|
||||
for (const options of [{ maxEntries: 129 }, { maxEntries: 0 }, { ttlMs: 0 }, { ttlMs: Infinity }]) {
|
||||
assert.throws(() => new WikipediaEnrichment(options), RangeError);
|
||||
}
|
||||
});
|
||||
|
||||
test('negative cache expires after 30 seconds; unavailable views retry after five minutes', async () => {
|
||||
let now = JANUARY.getTime();
|
||||
let failPage = true, failViews = true;
|
||||
const mock = mockFetch(url => {
|
||||
if (url.hostname === 'en.wikipedia.org') return failPage ? json({}) : json(metadata());
|
||||
return failViews ? new Response('', { status: 503 }) : json(views());
|
||||
});
|
||||
const service = new WikipediaEnrichment({ fetch: mock.fetcher, now: () => new Date(now) });
|
||||
await assert.rejects(service.get(736));
|
||||
failPage = false;
|
||||
now += 29_999;
|
||||
await assert.rejects(service.get(736));
|
||||
assert.equal(mock.calls.length, 1);
|
||||
now++;
|
||||
assert.equal((await service.get(736)).pageviews.status, 'unavailable');
|
||||
failViews = false;
|
||||
now += 299_999;
|
||||
assert.equal((await service.get(736)).pageviews.status, 'unavailable');
|
||||
assert.equal(mock.calls.length, 3);
|
||||
now++;
|
||||
assert.equal((await service.get(736)).pageviews.status, 'available');
|
||||
assert.equal(mock.calls.length, 5);
|
||||
});
|
||||
|
||||
test('cache rolls over at UTC month boundaries even before its TTL', async () => {
|
||||
let now = new Date('2024-12-31T23:59:00Z');
|
||||
const { service, calls } = fixture({ now: () => now });
|
||||
await service.get(736);
|
||||
now = new Date('2025-01-01T00:00:00Z');
|
||||
const result = await service.get(736);
|
||||
assert.equal(calls.length, 4);
|
||||
assert.equal(result.pageviews.latestMonth, '2024-12');
|
||||
assert.equal(result.pageviews.status, 'available');
|
||||
});
|
||||
|
||||
test('four concurrent jobs, bounded pending requests, busy rejection without a queue and recovery', async () => {
|
||||
const releases: (() => void)[] = [];
|
||||
const mock = mockFetch(async url => {
|
||||
if (url.hostname !== 'en.wikipedia.org') return json(views());
|
||||
await new Promise<void>(resolve => releases.push(resolve));
|
||||
return json(metadata(Number(url.searchParams.get('pageids'))));
|
||||
});
|
||||
const service = new WikipediaEnrichment({ fetch: mock.fetcher, now: () => JANUARY });
|
||||
const jobs = [1, 2, 3, 4].map(id => service.get(id));
|
||||
const duplicate = service.get(1);
|
||||
for (let id = 5; id <= 20; id++) await assert.rejects(service.get(id), /busy/);
|
||||
assert.equal(mock.calls.length, 4);
|
||||
releases.splice(0).forEach(release => release());
|
||||
await Promise.all([...jobs, duplicate]);
|
||||
assert.equal(mock.calls.length, 8);
|
||||
const next = service.get(5);
|
||||
releases.splice(0).forEach(release => release());
|
||||
await next;
|
||||
assert.equal(mock.calls.length, 10);
|
||||
});
|
||||
|
||||
test('JSON-only response contracts, redirects, announced and streamed one-MiB byte limits', async () => {
|
||||
const badResponses = [
|
||||
() => new Response('{}', { headers: { 'Content-Type': 'text/html' } }),
|
||||
() => new Response('{}'),
|
||||
() => new Response('{', { headers: { 'Content-Type': 'application/json' } }),
|
||||
() => new Response(null, { status: 302, headers: { Location: 'https://evil.example' } }),
|
||||
() => new Response('{}', { headers: { 'Content-Type': 'application/json', 'Content-Length': '1048577' } }),
|
||||
() => new Response('é'.repeat(600_000), { headers: { 'Content-Type': 'application/json' } }),
|
||||
() => { const response = json(metadata()); Object.defineProperty(response, 'redirected', { value: true }); return response; },
|
||||
];
|
||||
for (const makeResponse of badResponses) {
|
||||
const mock = mockFetch(() => makeResponse());
|
||||
await assert.rejects(new WikipediaEnrichment({ fetch: mock.fetcher }).get(736), /unavailable/);
|
||||
assert.equal(mock.calls.length, 1);
|
||||
}
|
||||
let cancelled = false;
|
||||
const mock = mockFetch(() => new Response(new ReadableStream<Uint8Array>({
|
||||
pull(controller) { controller.enqueue(new Uint8Array(600_000)); },
|
||||
cancel() { cancelled = true; },
|
||||
}), { headers: { 'Content-Type': 'application/json' } }));
|
||||
await assert.rejects(new WikipediaEnrichment({ fetch: mock.fetcher }).get(736));
|
||||
assert.equal(cancelled, true);
|
||||
});
|
||||
|
||||
test('every fetch receives a six-second timeout and aborted views retain metadata', async t => {
|
||||
const signals: AbortController[] = [];
|
||||
t.mock.method(AbortSignal, 'timeout', (ms: number) => {
|
||||
assert.equal(ms, 6000);
|
||||
const controller = new AbortController();
|
||||
signals.push(controller);
|
||||
return controller.signal;
|
||||
});
|
||||
const mock = mockFetch((url, init) => {
|
||||
if (url.hostname === 'en.wikipedia.org') return json(metadata());
|
||||
return new Promise<Response>((_resolve, reject) => {
|
||||
init.signal!.addEventListener('abort', () => reject(init.signal!.reason), { once: true });
|
||||
signals[1].abort(new DOMException('Timeout', 'TimeoutError'));
|
||||
});
|
||||
});
|
||||
const result = await new WikipediaEnrichment({ fetch: mock.fetcher, now: () => JANUARY }).get(736);
|
||||
assert.equal(signals.length, 2);
|
||||
assert.equal(result.pageviews.status, 'unavailable');
|
||||
assert.equal(result.pageId, 736);
|
||||
});
|
||||
@@ -26,4 +26,4 @@ test('difficulty retargets deterministically at exact windows and is bounded',()
|
||||
const fast=blocks.map((b,i)=>({...b,createdAt:1000+i}));
|
||||
assert.ok(BigInt(`0x${expectedTarget(fast)}`)<BigInt(`0x${MAX_TARGET}`));
|
||||
assert.match(CHAIN_ID,/^[a-f0-9]{64}$/);
|
||||
});
|
||||
});
|
||||
|
||||
@@ -408,4 +408,4 @@ test('peer sync pages at most 32 blocks per pass, preserves cursor and skips kno
|
||||
assert.equal(engine.calls, 33); // The already-known first block was not revalidated.
|
||||
assert.deepEqual(ledger.snapshot().blocks, chain.blocks);
|
||||
assert.equal(CONSENSUS.maxTransactions, 64);
|
||||
});
|
||||
});
|
||||
|
||||
@@ -228,4 +228,4 @@ test('wallet persistence commits before restore resolves, isolates demo DB, retr
|
||||
if (originalNavigator) Object.defineProperty(globalThis, 'navigator', originalNavigator);
|
||||
else Reflect.deleteProperty(globalThis, 'navigator');
|
||||
}
|
||||
});
|
||||
});
|
||||
|
||||
Reference in New Issue
Block a user