Redesign repo structure and get domains list from multiple sources #44

piquark6046 · 2026-01-14T09:15:39Z

Fix #39
Fix #42
Fix #40

builder/source/utils/http-server.ts

…n path expression Co-authored-by: Copilot Autofix powered by AI <62310815+github-advanced-security[bot]@users.noreply.github.com>

builder/source/utils/http-server.ts

…n path expression Co-authored-by: Copilot Autofix powered by AI <62310815+github-advanced-security[bot]@users.noreply.github.com>

…list

…s list

Copilot

Pull request overview

This PR redesigns the repository structure to use npm workspaces and implements a new multi-source approach for fetching Ad-Shield domains. The changes address issues where domains were missing from the userscript's @match rules, causing the script to not work on certain websites.

Changes:

Reorganized repository into workspaces (builder and userscript) with separate package.json and tsconfig.json files
Implemented domain fetching from multiple sources: IAB sellers.json, AdGuard filter lists, and uBlock Origin filter lists
Added build caching mechanism to reduce build times during development
Created a development debug server with file watching for hot-reload during development

Reviewed changes

Copilot reviewed 24 out of 29 changed files in this pull request and generated 10 comments.

Show a summary per file

File	Description
package.json	Added npm workspaces configuration, updated build scripts to use workspace commands
tsconfig.json	Added path mappings for workspace aliases (@builder/, @userscript/, @root/, @reporoot/)
builder/package.json	New workspace package with build, debug, and lint scripts; includes dependencies for building
builder/tsconfig.json	TypeScript configuration extending root config for builder workspace
builder/source/build.ts	Main build logic with domain fetching, wildcard expansion, and esbuild configuration
builder/source/buildci.ts	CLI entry point for CI/CD builds with argument parsing
builder/source/debug.ts	Development server with file watching for automatic rebuilds
builder/source/cache.ts	Caching mechanism to store and load domains for faster development builds
builder/source/banner/index.ts	UserScript metadata banner generation with domain injection
builder/source/references/index.ts	Aggregates domains from multiple sources (IAB sellers, filter lists)
builder/source/references/iabsellers.ts	Fetches domains from Ad-Shield's IAB sellers.json
builder/source/references/filterslists.ts	Combines domains from AdGuard and uBlock Origin filter lists
builder/source/references/filterslists/ADG.ts	Extracts Ad-Shield domains from AdGuard filter lists
builder/source/references/filterslists/uBO.ts	Extracts Ad-Shield domains from uBlock Origin filter lists
builder/source/references/filterslists/keywords.ts	Defines Ad-Shield CDN domains to exclude from matching
builder/source/references/custom-defined.ts	Placeholder for custom domain definitions
builder/source/utils/http-server.ts	HTTP server for serving built userscripts during development
builder/source/utils/wildcard-suffix-converter.ts	Converts wildcard domain suffixes to concrete domain patterns
userscript/package.json	New workspace package for userscript with lint script
userscript/tsconfig.json	TypeScript configuration for userscript workspace
userscript/source/index.ts	Main userscript logic (moved from sources/src/index.ts)
userscript/source/interface.ts	Updated import path to reflect new structure
userscript/source/utils.ts	Utility function for counting common strings between arrays
userscript/source/as-weakmap.ts	Ad-Shield WeakMap detection logic (extracted from main file)
builder.ts	Removed old monolithic build script
sources/banner.txt	Removed old banner template file
sources/esbuild.inject.ts	Removed empty inject file
.gitignore	Added .buildcache to ignored files
.github/workflows/build.yml	Changed to run on all branches instead of ignoring main

💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.

Copilot · 2026-01-19T00:01:03Z

builder/source/build.ts

+      ConvertWildcardSuffixToRegexPattern(Domain).forEach(GeneratedPattern => MatchingDomains.add(GeneratedPattern))
+    }
+  }
+


There is trailing whitespace on this line. Consider removing it for consistency with code formatting standards.

Suggested change

Copilot · 2026-01-19T00:01:03Z

builder/source/banner/index.ts

+  BannerString += '// @run-at       document-start\n'
+  BannerString += '//\n'
+  BannerString += `// @description  ${Options.Description['en']}\n`
+


There is trailing whitespace on this line. Consider removing it for consistency with code formatting standards.

Suggested change

Copilot · 2026-01-19T00:01:04Z

builder/source/references/filterslists/uBO.ts

+  let StatrtingLine = -1
+  let EndingLine = -1
+  for (const [Index, Filter] of AGTreeFiltersList.children.entries()) {
+    if (Filter.category === 'Comment' && typeof Filter.raws.text === 'string' && Filter.raws.text.includes(UBOFilterListAdShieldKeys.Starting)) {
+      StatrtingLine = Index
+    } else if (Filter.category === 'Comment' && typeof Filter.raws.text === 'string' && Filter.raws.text.includes(UBOFilterListAdShieldKeys.Ending)) {
+      EndingLine = Index
+    } else if (StatrtingLine !== -1 && EndingLine !== -1) {
+      break
+    } else if (Index === AGTreeFiltersList.children.length - 1) {
+      throw new Error('Could not find Ad-Shield ad reinsertion section in ' + UBOFilterListSpecificURL)
+    }
+  }
+  const AdShieldFilters = AGTreeFiltersList.children.filter((Filter, Index) => Index > StatrtingLine && Index < EndingLine)


The variable name "StatrtingLine" is misspelled. It should be "StartingLine".

Suggested change

let StatrtingLine = -1

let EndingLine = -1

for (const [Index, Filter] of AGTreeFiltersList.children.entries()) {

if (Filter.category === 'Comment' && typeof Filter.raws.text === 'string' && Filter.raws.text.includes(UBOFilterListAdShieldKeys.Starting)) {

StatrtingLine = Index

} else if (Filter.category === 'Comment' && typeof Filter.raws.text === 'string' && Filter.raws.text.includes(UBOFilterListAdShieldKeys.Ending)) {

EndingLine = Index

} else if (StatrtingLine !== -1 && EndingLine !== -1) {

break

} else if (Index === AGTreeFiltersList.children.length - 1) {

throw new Error('Could not find Ad-Shield ad reinsertion section in ' + UBOFilterListSpecificURL)

}

}

const AdShieldFilters = AGTreeFiltersList.children.filter((Filter, Index) => Index > StatrtingLine && Index < EndingLine)

let StartingLine = -1

let EndingLine = -1

for (const [Index, Filter] of AGTreeFiltersList.children.entries()) {

if (Filter.category === 'Comment' && typeof Filter.raws.text === 'string' && Filter.raws.text.includes(UBOFilterListAdShieldKeys.Starting)) {

StartingLine = Index

} else if (Filter.category === 'Comment' && typeof Filter.raws.text === 'string' && Filter.raws.text.includes(UBOFilterListAdShieldKeys.Ending)) {

EndingLine = Index

} else if (StartingLine !== -1 && EndingLine !== -1) {

break

} else if (Index === AGTreeFiltersList.children.length - 1) {

throw new Error('Could not find Ad-Shield ad reinsertion section in ' + UBOFilterListSpecificURL)

}

}

const AdShieldFilters = AGTreeFiltersList.children.filter((Filter, Index) => Index > StartingLine && Index < EndingLine)

Copilot · 2026-01-19T00:01:04Z

builder/source/references/filterslists/ADG.ts

+  let StatrtingLine = -1
+  let EndingLine = -1
+  for (const [Index, Filter] of AGTreeFiltersList.children.entries()) {
+    if (Filter.category === 'Comment' && typeof Filter.raws.text === 'string' && Filter.raws.text.includes(AGBaseFilterListAdShieldKeys.Starting)) {
+      StatrtingLine = Index
+    } else if (Filter.category === 'Comment' && typeof Filter.raws.text === 'string' && Filter.raws.text.includes(AGBaseFilterListAdShieldKeys.Ending)) {
+      EndingLine = Index
+    } else if (StatrtingLine !== -1 && EndingLine !== -1) {
+      break
+    } else if (Index === AGTreeFiltersList.children.length - 1) {
+      throw new Error('Could not find Ad-Shield ad reinsertion section in ' + AGBaseFilterListSpecificURL)
+    }
+  }
+  const AdShieldFilters = AGTreeFiltersList.children.filter((Filter, Index) => Index > StatrtingLine && Index < EndingLine)


The variable name "StatrtingLine" is misspelled. It should be "StartingLine".

Suggested change

let StatrtingLine = -1

let EndingLine = -1

for (const [Index, Filter] of AGTreeFiltersList.children.entries()) {

if (Filter.category === 'Comment' && typeof Filter.raws.text === 'string' && Filter.raws.text.includes(AGBaseFilterListAdShieldKeys.Starting)) {

StatrtingLine = Index

} else if (Filter.category === 'Comment' && typeof Filter.raws.text === 'string' && Filter.raws.text.includes(AGBaseFilterListAdShieldKeys.Ending)) {

EndingLine = Index

} else if (StatrtingLine !== -1 && EndingLine !== -1) {

break

} else if (Index === AGTreeFiltersList.children.length - 1) {

throw new Error('Could not find Ad-Shield ad reinsertion section in ' + AGBaseFilterListSpecificURL)

}

}

const AdShieldFilters = AGTreeFiltersList.children.filter((Filter, Index) => Index > StatrtingLine && Index < EndingLine)

let StartingLine = -1

let EndingLine = -1

for (const [Index, Filter] of AGTreeFiltersList.children.entries()) {

if (Filter.category === 'Comment' && typeof Filter.raws.text === 'string' && Filter.raws.text.includes(AGBaseFilterListAdShieldKeys.Starting)) {

StartingLine = Index

} else if (Filter.category === 'Comment' && typeof Filter.raws.text === 'string' && Filter.raws.text.includes(AGBaseFilterListAdShieldKeys.Ending)) {

EndingLine = Index

} else if (StartingLine !== -1 && EndingLine !== -1) {

break

} else if (Index === AGTreeFiltersList.children.length - 1) {

throw new Error('Could not find Ad-Shield ad reinsertion section in ' + AGBaseFilterListSpecificURL)

}

}

const AdShieldFilters = AGTreeFiltersList.children.filter((Filter, Index) => Index > StartingLine && Index < EndingLine)

Copilot · 2026-01-19T00:01:04Z

builder/source/references/iabsellers.ts

+
+export async function FetchIABSellersJsonData(): Promise<string[]> {
+  const IABSellersJsonResponse: { StatusCode: number, Headers: Record<string, string | string[]>, Body: unknown } = await HTTPSRequest(new URL(IABSellersJsonURL), { ExpectedAs: 'JSON' })
+  let IABSellersJsonData =IABSellersJsonResponse.Body as {


Missing space after the equals sign. Should be "let IABSellersJsonData = IABSellersJsonResponse.Body".

Suggested change

let IABSellersJsonData =IABSellersJsonResponse.Body as {

let IABSellersJsonData = IABSellersJsonResponse.Body as {

Copilot · 2026-01-19T00:01:04Z

builder/source/debug.ts

+const WatchingGlob = [];
+['builder/', 'userscript/', ''].forEach(Dir => {
+  WatchingGlob.push(`${ProcessCwd}/${Dir}sources/**/*.ts`)
+  WatchingGlob.push(`${ProcessCwd}/${Dir}sources/**/*.json`)
+  WatchingGlob.push(`${ProcessCwd}/${Dir}sources/**/*.txt`)
+})


The WatchingGlob array is constructed but never used. The Watcher uses hardcoded paths instead of the WatchingGlob. Either use WatchingGlob in the Chokidar.watch() call or remove it.

Suggested change

const WatchingGlob = [];

['builder/', 'userscript/', ''].forEach(Dir => {

WatchingGlob.push(`${ProcessCwd}/${Dir}sources/**/*.ts`)

WatchingGlob.push(`${ProcessCwd}/${Dir}sources/**/*.json`)

WatchingGlob.push(`${ProcessCwd}/${Dir}sources/**/*.txt`)

})

Copilot · 2026-01-19T00:01:05Z

builder/source/buildci.ts

+  Minify: Zod.string().pipe(Zod.enum(['true', 'false'])).transform(Value => Value === 'true').default(true),
+  UseCache: Zod.string().pipe(Zod.enum(['true', 'false'])).transform(Value => Value === 'true').default(true),


The .default(true) method expects a boolean value, but it's being applied to a string schema before the transform. This should be .default('true') since the schema expects a string input that gets transformed to a boolean.

Suggested change

Minify: Zod.string().pipe(Zod.enum(['true', 'false'])).transform(Value => Value === 'true').default(true),

UseCache: Zod.string().pipe(Zod.enum(['true', 'false'])).transform(Value => Value === 'true').default(true),

Minify: Zod.string().pipe(Zod.enum(['true', 'false'])).transform(Value => Value === 'true').default('true'),

UseCache: Zod.string().pipe(Zod.enum(['true', 'false'])).transform(Value => Value === 'true').default('true'),

Copilot · 2026-01-19T00:01:05Z

builder/source/utils/http-server.ts

+    } else if (!IsLoopBack(Req.socket.remoteAddress)) {
+      Res.writeHead(403)
+      Res.end()
+      return
+    } else if (ShouldPreventHTTPResponse || !Fs.existsSync(ResolvedPath)) {


The expression Req.socket.remoteAddress may be undefined, which would cause IsLoopBack to receive undefined. Consider adding a null check or providing a default value before passing to IsLoopBack.

Suggested change

} else if (!IsLoopBack(Req.socket.remoteAddress)) {

Res.writeHead(403)

Res.end()

return

} else if (ShouldPreventHTTPResponse || !Fs.existsSync(ResolvedPath)) {

} else {

const RemoteAddress = Req.socket.remoteAddress

if (!RemoteAddress || !IsLoopBack(RemoteAddress)) {

Res.writeHead(403)

Res.end()

return

}

}

if (ShouldPreventHTTPResponse || !Fs.existsSync(ResolvedPath)) {

Copilot · 2026-01-19T00:01:05Z

builder/source/debug.ts

+import { Build } from './build.js'
+
+const ProcessCwd = Process.cwd()
+const WatchingGlob = [];


The WatchingGlob array is declared without a type annotation. Consider adding an explicit type annotation: "const WatchingGlob: string[] = [];".

Suggested change

const WatchingGlob = [];

const WatchingGlob: string[] = [];

Copilot · 2026-01-19T00:01:06Z

builder/package.json

+    "@typescriptprime/parsing": "^1.0.4",
+    "@typescriptprime/securereq": "^1.1.0",
+    "chokidar": "^5.0.0",
+    "esbuild": "^0.27.2",


The esbuild version 0.27.2 appears to be outdated. As of January 2025, esbuild was at version 0.24.x or higher. Consider updating to a more recent version.

Suggested change

"esbuild": "^0.27.2",

"esbuild": "^0.24.0",

piquark6046 added 2 commits January 14, 2026 09:13

refactor: design monorepos

3eb385d

ci: fix ESLint chain

1b9724c

piquark6046 changed the title ~~Resign repo structure and get domains list from multiple sources~~ Redesign repo structure and get domains list from multiple sources Jan 14, 2026

piquark6046 added 6 commits January 14, 2026 09:33

ci: expand ESLint checking to all Typescript files

2973ddc

refactor: split IAB sellers downloader

f2bf1af

refactor: update pacakge.json in builder

0173bdb

fix: baseUrl in tsconfig.json leads fatal of tsx execution

b7c2fd8

chore: add http-server utils

45468db

chore: update .gitignore and package.json for new monorepo structure

e4dcb6a

github-advanced-security bot found potential problems Jan 16, 2026

View reviewed changes

builder/source/utils/http-server.ts Fixed Show fixed Hide fixed

builder/source/utils/http-server.ts Fixed Show fixed Hide fixed

Potential fix for code scanning alert no. 4: Uncontrolled data used i…

f2532bd

…n path expression Co-authored-by: Copilot Autofix powered by AI <62310815+github-advanced-security[bot]@users.noreply.github.com>

github-advanced-security bot found potential problems Jan 18, 2026

View reviewed changes

builder/source/utils/http-server.ts Fixed Show fixed Hide fixed

builder/source/utils/http-server.ts Fixed Show fixed Hide fixed

chore: follow PascalCase

f6f448f

github-advanced-security bot found potential problems Jan 18, 2026

View reviewed changes

builder/source/utils/http-server.ts Fixed Show fixed Hide fixed

builder/source/utils/http-server.ts Fixed Show fixed Hide fixed

piquark6046 and others added 14 commits January 18, 2026 21:26

Potential fix for code scanning alert no. 9: Uncontrolled data used i…

785d139

…n path expression Co-authored-by: Copilot Autofix powered by AI <62310815+github-advanced-security[bot]@users.noreply.github.com>

feat: add IndexAdShieldDomainsFromAG func to refer AG Base filters …

57f8f26

…list

feat: add checking func to discard all CDN domains owned by Ad-Shield

5d8fdcf

feat: add IndexAdShieldDomainsFromUBO func to refer uBO filters list

0b7640a

feat: add FetchAdShieldDomains func to refer AG Base and uBO filter…

c0bf75c

…s list

chore: remove unnecessary import

b253d9b

ci: run ESLint workflow for all branches

4be8bd9

chore: rename FetchAdShieldDomainsFromFiltersLists

5a5ad11

feat: add FetchAdShieldDomains func to refer all references

79f1ab5

feat: provide CustomDefinedMatches Set

5dab3b1

feat: add building cache for caching remote domains

b1c9c84

feat: create CreateBanner func

9fb4d9f

fix: path is not resolved correctly if npm run is used

33f3ae6

feat: use new builder

3b37419

piquark6046 marked this pull request as ready for review January 18, 2026 23:56

Copilot AI review requested due to automatic review settings January 18, 2026 23:56

Copilot started reviewing on behalf of piquark6046 January 18, 2026 23:56 View session

Copilot AI reviewed Jan 19, 2026

View reviewed changes

piquark6046 added 3 commits January 19, 2026 00:19

chore: add SubscriptionUrl customization for debugging

4f8b980

fix: chokidar does not built-in glob support

efef1c4

fix: typo

b12ec19

piquark6046 merged commit 169eaf3 into main Jan 19, 2026
7 checks passed

	let IABSellersJsonData =IABSellersJsonResponse.Body as {
	let IABSellersJsonData = IABSellersJsonResponse.Body as {

		Minify: Zod.string().pipe(Zod.enum(['true', 'false'])).transform(Value => Value === 'true').default(true),
		UseCache: Zod.string().pipe(Zod.enum(['true', 'false'])).transform(Value => Value === 'true').default(true),

Redesign repo structure and get domains list from multiple sources #44

Redesign repo structure and get domains list from multiple sources #44

Conversation

piquark6046 commented Jan 14, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Copilot AI left a comment

Choose a reason for hiding this comment

Pull request overview

Reviewed changes

Uh oh!

Copilot AI Jan 19, 2026

Choose a reason for hiding this comment

Uh oh!

Copilot AI Jan 19, 2026

Choose a reason for hiding this comment

Uh oh!

Copilot AI Jan 19, 2026

Choose a reason for hiding this comment

Uh oh!

Copilot AI Jan 19, 2026

Choose a reason for hiding this comment

Uh oh!

Copilot AI Jan 19, 2026

Choose a reason for hiding this comment

Uh oh!

Copilot AI Jan 19, 2026

Choose a reason for hiding this comment

Uh oh!

Copilot AI Jan 19, 2026

Choose a reason for hiding this comment

Uh oh!

Copilot AI Jan 19, 2026

Choose a reason for hiding this comment

Uh oh!

Copilot AI Jan 19, 2026

Choose a reason for hiding this comment

Uh oh!

Copilot AI Jan 19, 2026

Choose a reason for hiding this comment

Uh oh!

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

2 participants

piquark6046 commented Jan 14, 2026 •

edited

Loading