Compare commits

..
Author SHA1 Message Date
shamoon 3b36dcf096 Tweak to allow the x to remove the whole badge 2026-11-21 11:30:33 -08:00
shamoon debc48551d Coverage 2026-11-21 11:17:54 -08:00
shamoon 3e63b803e9 Oops, left that 2026-11-21 11:15:05 -08:00
shamoon b1bb4bdf84 Fix backspace bug 2026-11-21 10:39:47 -08:00
shamoon f6b8055150 Wire in the collapseTags, split and parentClick. Update get more tags too 2026-11-21 10:33:37 -08:00
shamoon f95abddaf4 Use the ancestor helper and switch based on the setting in tags component 2026-11-21 10:29:36 -08:00
shamoon 4ef982e9dc actual css animation stuff for the tag and hover, ancestor parsing and parent click output 2026-11-21 10:26:56 -08:00
shamoon 8cfd6bae42 Add collapse tags helper and the pipe 2026-11-21 10:00:43 -08:00
shamoon 613331740c Add a setting for this 2026-11-21 10:00:43 -08:00
shamoon 14252054a1 Fix/tweak: update form inputs while typing (#14821) 2026-11-18 16:28:02 -08:00
github-actions[bot]andshamoon bcfdfcaa29 Documentation: Add v3.5.1 changelog (#14814)
* Changelog v3.5.1 - GHA

* Modify changelog for paperless-ngx version 3.5.1

Updated changelog to reflect changes in version 3.5.1.

---------

Co-authored-by: github-actions <41898282+github-actions[bot]@users.noreply.github.com>
Co-authored-by: shamoon <4887959+shamoon@users.noreply.github.com>
2026-11-17 21:52:43 -08:00
shamoon 35b0439195 Bump version to 3.5.1 2026-11-17 21:05:28 -08:00
github-actions[bot]andCrowdin Bot f21f6b9795 New Crowdin translations by GitHub Action (#14798)
Co-authored-by: Crowdin Bot <support+bot@crowdin.com>
2026-11-18 04:15:05 +00:00
Trenton H 7977093e83 Chore: restrict uv lock environments to supported architectures (#14759)
* Chore: restrict uv lock environments to supported architectures

Limit the lock to Linux x86_64, Linux aarch64 and macOS arm64. Locked
versions are unchanged; only the unused wheel entries are removed.

* Chore: restrict uv lock environments to CPython

Add platform_python_implementation == 'CPython' to each supported
environment so the lock only resolves for interpreters we actually run on.
Basically PyPy was never supported anyway
2026-11-17 21:16:40 +00:00
shamoon 237521ec4b Fix: skip dont use set_actor if audit log disabled (#14808) 2026-11-17 21:02:16 +00:00
GitHub Actions 0596579d4b Auto translate strings 2026-11-17 20:49:24 +00:00
shamoon a22579e311 Tweak: adjust chat timing for streaming (#14805) 2026-11-17 20:47:55 +00:00
Trenton H 199d1a88e9 Fix: stream AI chat answers under ASGI (#14803) 2026-11-17 20:47:55 +00:00
GitHub Actions f68c093097 Auto translate strings 2026-11-17 11:38:40 +00:00
shamoon b5d6009e15 Tweak/fix: retain bulk edit selection after queuing operation (#14797) 2026-11-17 03:37:03 -08:00
github-actions[bot] c3fa454ffa Changelog v3.5.0 - GHA (#14792)
Co-authored-by: github-actions <41898282+github-actions[bot]@users.noreply.github.com>
2026-11-16 19:48:40 -08:00
shamoon bfd0d7fe36 Bump version to 3.5.0 2026-11-16 19:20:01 -08:00
github-actions[bot]andCrowdin Bot 48c7a11195 New Crowdin translations by GitHub Action (#14761)
Co-authored-by: Crowdin Bot <support+bot@crowdin.com>
2026-11-17 03:11:47 +00:00
GitHub Actions 9ba98d1cb3 Auto translate strings 2026-11-17 00:25:18 +00:00
shamoon de75249e51 Chorehancement: move all bulk operations to async hand-off (#14773) 2026-11-17 00:23:44 +00:00
GitHub Actions 28bb63824f Auto translate strings 2026-11-16 15:29:38 +00:00
Trenton H 64bf7f8457 Fix: use a constant password mask and consolidate the no-change check (#14784)
Return a constant 10 character mask instead.

Move ObfuscatedPasswordField into paperless/fields.py so core no longer
imports it from paperless_mail, and add is_obfuscated_password() to
replace the five checks
2026-11-16 15:28:13 +00:00
Trenton H c177c13acf Fix: use a PostgreSQL-compatible regex for scheduled workflow filename filters (#14783)
* Fix: use a PostgreSQL-compatible regex for scheduled workflow filename filters

* Tests for the translation layer
2026-11-16 06:42:34 -08:00
164 changed files with 27814 additions and 14303 deletions

No files matched your search

+7 -7
View File
@@ -285,14 +285,14 @@ The API supports various bulk-editing operations which are executed asynchronous
### Documents
Document bulk operations return `{"result":"OK","task_id":"…"}` when queued.
Check `/api/tasks/?task_id={uuid}` for the outcome, including any follow-up processing,
deletion, or recovery. Failed operations may leave partial changes applied.
Document bulk operations, including the dedicated document action endpoints, return
`{"result":"OK","task_id":"…"}` when queued. Use `/api/tasks/?task_id={uuid}`
to check the task outcome. Actions that create or reprocess files may start further
consumption or reprocessing tasks and completion of the bulk task means those tasks
were scheduled, not that their file processing has finished.
For operations with child tasks, `result_data` includes `succeeded_count` and
`failed_count` (task counts). Merges report skipped inputs in `failed_document_ids`.
The `/api/documents/bulk_edit/` endpoint accepts:
For bulk operations on documents, use the endpoint `/api/documents/bulk_edit/` which accepts
a json payload of the format:
```json
{
+116
View File
@@ -1,5 +1,121 @@
# Changelog
## paperless-ngx 3.5.1
### Changes
- Tweak: adjust chat timing for streaming [@shamoon](https://github.com/shamoon) ([#14805](https://github.com/paperless-ngx/paperless-ngx/pull/14805))
### Bug Fixes
- Fix: dont use set\_actor if audit log disabled in tasks [@shamoon](https://github.com/shamoon) ([#14808](https://github.com/paperless-ngx/paperless-ngx/pull/14808))
- Fix: stream AI chat answers under ASGI [@stumpylog](https://github.com/stumpylog) ([#14803](https://github.com/paperless-ngx/paperless-ngx/pull/14803))
- Tweak/fix: retain bulk edit selection after queuing operation [@shamoon](https://github.com/shamoon) ([#14797](https://github.com/paperless-ngx/paperless-ngx/pull/14797))
### All App Changes
<details>
<summary>4 changes</summary>
- Fix: dont use set\_actor if audit log disabled in tasks [@shamoon](https://github.com/shamoon) ([#14808](https://github.com/paperless-ngx/paperless-ngx/pull/14808))
- Tweak: adjust chat timing for streaming [@shamoon](https://github.com/shamoon) ([#14805](https://github.com/paperless-ngx/paperless-ngx/pull/14805))
- Fix: stream AI chat answers under ASGI [@stumpylog](https://github.com/stumpylog) ([#14803](https://github.com/paperless-ngx/paperless-ngx/pull/14803))
- Tweak/fix: retain bulk edit selection after queuing operation [@shamoon](https://github.com/shamoon) ([#14797](https://github.com/paperless-ngx/paperless-ngx/pull/14797))
</details>
## paperless-ngx 3.5.0
### Features / Enhancements
- Chorehancement: move all bulk operations to async hand-off [@shamoon](https://github.com/shamoon) ([#14773](https://github.com/paperless-ngx/paperless-ngx/pull/14773))
- Enhancement: render document notes as markdown [@zwif](https://github.com/zwif) ([#14626](https://github.com/paperless-ngx/paperless-ngx/pull/14626))
- Feature: manual workflow trigger [@shamoon](https://github.com/shamoon) ([#14704](https://github.com/paperless-ngx/paperless-ngx/pull/14704))
- Enhancement: make bulk metadata operations run async [@shamoon](https://github.com/shamoon) ([#14711](https://github.com/paperless-ngx/paperless-ngx/pull/14711))
- Fixhancement: add automatic clearing of sessions [@shamoon](https://github.com/shamoon) ([#14722](https://github.com/paperless-ngx/paperless-ngx/pull/14722))
- Tweak: chat reference tweaks [@shamoon](https://github.com/shamoon) ([#14723](https://github.com/paperless-ngx/paperless-ngx/pull/14723))
- Fixhancement: rework saved view widget queue processing to expected [@shamoon](https://github.com/shamoon) ([#14699](https://github.com/paperless-ngx/paperless-ngx/pull/14699))
### Bug Fixes
- Fix: use a constant password mask and consolidate the no-change check [@stumpylog](https://github.com/stumpylog) ([#14784](https://github.com/paperless-ngx/paperless-ngx/pull/14784))
- Fix: use a PostgreSQL-compatible regex for scheduled workflow filename filters [@stumpylog](https://github.com/stumpylog) ([#14783](https://github.com/paperless-ngx/paperless-ngx/pull/14783))
- Fix: get users for saved view create dialog [@shamoon](https://github.com/shamoon) ([#14769](https://github.com/paperless-ngx/paperless-ngx/pull/14769))
- Fix: Allow AI chat filters over more than 32700 documents [@stumpylog](https://github.com/stumpylog) ([#14754](https://github.com/paperless-ngx/paperless-ngx/pull/14754))
- Fix: Enforce CRLF when writing an email message to the file [@stumpylog](https://github.com/stumpylog) ([#14745](https://github.com/paperless-ngx/paperless-ngx/pull/14745))
- Fix: restore image without archive version thumbnail generation [@shamoon](https://github.com/shamoon) ([#14741](https://github.com/paperless-ngx/paperless-ngx/pull/14741))
- Fix: Blacklist certain OCR user arguments that paperless itself uses [@stumpylog](https://github.com/stumpylog) ([#14734](https://github.com/paperless-ngx/paperless-ngx/pull/14734))
- Fix: remove redundant ui task text filters [@shamoon](https://github.com/shamoon) ([#14736](https://github.com/paperless-ngx/paperless-ngx/pull/14736))
- Fix: fix allowed\_hosts default [@shamoon](https://github.com/shamoon) ([#14735](https://github.com/paperless-ngx/paperless-ngx/pull/14735))
- Fixhancement: add automatic clearing of sessions [@shamoon](https://github.com/shamoon) ([#14722](https://github.com/paperless-ngx/paperless-ngx/pull/14722))
- Fix: two dropdown-related UI fixes [@shamoon](https://github.com/shamoon) ([#14717](https://github.com/paperless-ngx/paperless-ngx/pull/14717))
- Fix: use filtered documents for statistics ASN max [@shamoon](https://github.com/shamoon) ([#14710](https://github.com/paperless-ngx/paperless-ngx/pull/14710))
- Fix: make config view permissions consistent [@shamoon](https://github.com/shamoon) ([#14709](https://github.com/paperless-ngx/paperless-ngx/pull/14709))
- Fix: release the LLM index read lock before streaming the chat answer [@stumpylog](https://github.com/stumpylog) ([#14674](https://github.com/paperless-ngx/paperless-ngx/pull/14674))
- Fix: retrieve once per document chat question [@stumpylog](https://github.com/stumpylog) ([#14673](https://github.com/paperless-ngx/paperless-ngx/pull/14673))
- Fixhancement: rework saved view widget queue processing to expected [@shamoon](https://github.com/shamoon) ([#14699](https://github.com/paperless-ngx/paperless-ngx/pull/14699))
- Fix: apply PDF/A conversion once for mail archives [@shamoon](https://github.com/shamoon) ([#14698](https://github.com/paperless-ngx/paperless-ngx/pull/14698))
- Fix default MFA trusted AMR setting [@shamoon](https://github.com/shamoon) ([#14693](https://github.com/paperless-ngx/paperless-ngx/pull/14693))
- Fix: check descendant tag permissions on delete [@shamoon](https://github.com/shamoon) ([#14687](https://github.com/paperless-ngx/paperless-ngx/pull/14687))
- Fix: lock rows for first-user signup [@shamoon](https://github.com/shamoon) ([#14688](https://github.com/paperless-ngx/paperless-ngx/pull/14688))
- Fix: prevent ID badge wrapping under title on mobile [@shamoon](https://github.com/shamoon) ([#14682](https://github.com/paperless-ngx/paperless-ngx/pull/14682))
### Maintenance
- Chore(deps): Bump the actions group across 1 directory with 17 updates @[dependabot[bot]](https://github.com/apps/dependabot) ([#14691](https://github.com/paperless-ngx/paperless-ngx/pull/14691))
### Dependencies
<details>
<summary>4 changes</summary>
- Chore(deps): Bump the pre-commit-dependencies group with 4 updates @[dependabot[bot]](https://github.com/apps/dependabot) ([#14636](https://github.com/paperless-ngx/paperless-ngx/pull/14636))
- Chore(deps): Bump the actions group across 1 directory with 17 updates @[dependabot[bot]](https://github.com/apps/dependabot) ([#14691](https://github.com/paperless-ngx/paperless-ngx/pull/14691))
- docker(deps): bump astral-sh/uv from 0.13.5-python3.14-trixie-slim to 0.13.7-python3.14-trixie-slim @[dependabot[bot]](https://github.com/apps/dependabot) ([#14635](https://github.com/paperless-ngx/paperless-ngx/pull/14635))
- chore(deps): bump django-allauth to 65.21.1 [@stumpylog](https://github.com/stumpylog) ([#14672](https://github.com/paperless-ngx/paperless-ngx/pull/14672))
</details>
### All App Changes
<details>
<summary>32 changes</summary>
- Chorehancement: move all bulk operations to async hand-off [@shamoon](https://github.com/shamoon) ([#14773](https://github.com/paperless-ngx/paperless-ngx/pull/14773))
- Fix: use a constant password mask and consolidate the no-change check [@stumpylog](https://github.com/stumpylog) ([#14784](https://github.com/paperless-ngx/paperless-ngx/pull/14784))
- Fix: use a PostgreSQL-compatible regex for scheduled workflow filename filters [@stumpylog](https://github.com/stumpylog) ([#14783](https://github.com/paperless-ngx/paperless-ngx/pull/14783))
- Fix: get users for saved view create dialog [@shamoon](https://github.com/shamoon) ([#14769](https://github.com/paperless-ngx/paperless-ngx/pull/14769))
- Enhancement: render document notes as markdown [@zwif](https://github.com/zwif) ([#14626](https://github.com/paperless-ngx/paperless-ngx/pull/14626))
- Feature: manual workflow trigger [@shamoon](https://github.com/shamoon) ([#14704](https://github.com/paperless-ngx/paperless-ngx/pull/14704))
- Fix: Allow AI chat filters over more than 32700 documents [@stumpylog](https://github.com/stumpylog) ([#14754](https://github.com/paperless-ngx/paperless-ngx/pull/14754))
- Fix: Enforce CRLF when writing an email message to the file [@stumpylog](https://github.com/stumpylog) ([#14745](https://github.com/paperless-ngx/paperless-ngx/pull/14745))
- Enhancement: make bulk metadata operations run async [@shamoon](https://github.com/shamoon) ([#14711](https://github.com/paperless-ngx/paperless-ngx/pull/14711))
- Performance: load a cached Hugging Face embedding model without contacting the Hub [@stumpylog](https://github.com/stumpylog) ([#14728](https://github.com/paperless-ngx/paperless-ngx/pull/14728))
- Fix: restore image without archive version thumbnail generation [@shamoon](https://github.com/shamoon) ([#14741](https://github.com/paperless-ngx/paperless-ngx/pull/14741))
- Fix: Blacklist certain OCR user arguments that paperless itself uses [@stumpylog](https://github.com/stumpylog) ([#14734](https://github.com/paperless-ngx/paperless-ngx/pull/14734))
- Fix: remove redundant ui task text filters [@shamoon](https://github.com/shamoon) ([#14736](https://github.com/paperless-ngx/paperless-ngx/pull/14736))
- Fix: fix allowed\_hosts default [@shamoon](https://github.com/shamoon) ([#14735](https://github.com/paperless-ngx/paperless-ngx/pull/14735))
- Fixhancement: add automatic clearing of sessions [@shamoon](https://github.com/shamoon) ([#14722](https://github.com/paperless-ngx/paperless-ngx/pull/14722))
- Tweak: chat reference tweaks [@shamoon](https://github.com/shamoon) ([#14723](https://github.com/paperless-ngx/paperless-ngx/pull/14723))
- Fix: two dropdown-related UI fixes [@shamoon](https://github.com/shamoon) ([#14717](https://github.com/paperless-ngx/paperless-ngx/pull/14717))
- Fix: use filtered documents for statistics ASN max [@shamoon](https://github.com/shamoon) ([#14710](https://github.com/paperless-ngx/paperless-ngx/pull/14710))
- Fix: make config view permissions consistent [@shamoon](https://github.com/shamoon) ([#14709](https://github.com/paperless-ngx/paperless-ngx/pull/14709))
- Performance: stop embedding every document ID in selection\_data queries [@stumpylog](https://github.com/stumpylog) ([#14597](https://github.com/paperless-ngx/paperless-ngx/pull/14597))
- Performance: replace django-treenode with adjacency-list tags [@shamoon](https://github.com/shamoon) ([#14612](https://github.com/paperless-ngx/paperless-ngx/pull/14612))
- Performance: use small vec0 chunks and larger pages for the LLM index [@stumpylog](https://github.com/stumpylog) ([#14678](https://github.com/paperless-ngx/paperless-ngx/pull/14678))
- Fix: release the LLM index read lock before streaming the chat answer [@stumpylog](https://github.com/stumpylog) ([#14674](https://github.com/paperless-ngx/paperless-ngx/pull/14674))
- Fix: retrieve once per document chat question [@stumpylog](https://github.com/stumpylog) ([#14673](https://github.com/paperless-ngx/paperless-ngx/pull/14673))
- Fixhancement: rework saved view widget queue processing to expected [@shamoon](https://github.com/shamoon) ([#14699](https://github.com/paperless-ngx/paperless-ngx/pull/14699))
- Fix: apply PDF/A conversion once for mail archives [@shamoon](https://github.com/shamoon) ([#14698](https://github.com/paperless-ngx/paperless-ngx/pull/14698))
- Fix default MFA trusted AMR setting [@shamoon](https://github.com/shamoon) ([#14693](https://github.com/paperless-ngx/paperless-ngx/pull/14693))
- Fix: check descendant tag permissions on delete [@shamoon](https://github.com/shamoon) ([#14687](https://github.com/paperless-ngx/paperless-ngx/pull/14687))
- Fix: lock rows for first-user signup [@shamoon](https://github.com/shamoon) ([#14688](https://github.com/paperless-ngx/paperless-ngx/pull/14688))
- Fix: prevent ID badge wrapping under title on mobile [@shamoon](https://github.com/shamoon) ([#14682](https://github.com/paperless-ngx/paperless-ngx/pull/14682))
- chore(deps): bump django-allauth to 65.21.1 [@stumpylog](https://github.com/stumpylog) ([#14672](https://github.com/paperless-ngx/paperless-ngx/pull/14672))
- chore(deps): bump dev tooling [@stumpylog](https://github.com/stumpylog) ([#14671](https://github.com/paperless-ngx/paperless-ngx/pull/14671))
</details>
## paperless-ngx 3.4.2
### Bug Fixes
+4 -3
View File
@@ -1,6 +1,6 @@
[project]
name = "paperless-ngx"
version = "3.4.2"
version = "3.5.1"
description = """\
A community-supported supercharged document management system: scan, index and archive all your physical documents\
"""
@@ -151,8 +151,9 @@ typing = [
required-version = ">=0.9.0"
prerelease = "disallow"
environments = [
"sys_platform == 'darwin'",
"sys_platform == 'linux'",
"sys_platform == 'darwin' and platform_machine == 'arm64' and platform_python_implementation == 'CPython'",
"sys_platform == 'linux' and platform_machine == 'aarch64' and platform_python_implementation == 'CPython'",
"sys_platform == 'linux' and platform_machine == 'x86_64' and platform_python_implementation == 'CPython'",
]
package = false
[tool.uv.sources]
+490 -249
View File
File diff suppressed because it is too large. Load diff
+1 -1
View File
@@ -1,6 +1,6 @@
{
"name": "paperless-ngx-ui",
"version": "3.4.2",
"version": "3.5.1",
"scripts": {
"preinstall": "npx only-allow pnpm",
"ng": "ng",
@@ -227,6 +227,12 @@
</div>
</div>
<div class="row">
<div class="col">
<pngx-input-check i18n-title title="Show nested tags with their parents" i18n-hint hint="Child tags are shown together with their parent tags, instead of separately." formControlName="tagsShowHierarchy"></pngx-input-check>
</div>
</div>
<h5 class="mt-3" i18n>Document editing</h5>
<div class="row">
<div class="col">
@@ -305,7 +305,7 @@ describe('SettingsComponent', () => {
expect(toastErrorSpy).toHaveBeenCalled()
expect(storeSpy).toHaveBeenCalled()
expect(appearanceSettingsSpy).not.toHaveBeenCalled()
expect(setSpy).toHaveBeenCalledTimes(33)
expect(setSpy).toHaveBeenCalledTimes(34)
expect(setSpy).toHaveBeenCalledWith(SETTINGS_KEYS.SIDEBAR_HIDDEN_ITEMS, [
HideableSidebarItemID.Workflows,
])
@@ -163,6 +163,7 @@ export class SettingsComponent
bulkEditConfirmationDialogs: new FormControl(null),
bulkEditApplyOnClose: new FormControl(null),
documentListItemPerPage: new FormControl(null),
tagsShowHierarchy: new FormControl(null),
slimSidebarEnabled: new FormControl(null),
sidebarHiddenItems: new FormControl<HideableSidebarItemID[]>([]),
darkModeUseSystem: new FormControl(null),
@@ -342,6 +343,7 @@ export class SettingsComponent
bulkEditApplyOnClose: this.settings.get(
SETTINGS_KEYS.BULK_EDIT_APPLY_ON_CLOSE
),
tagsShowHierarchy: this.settings.get(SETTINGS_KEYS.TAGS_SHOW_HIERARCHY),
documentListItemPerPage: this.settings.get(
SETTINGS_KEYS.DOCUMENT_LIST_SIZE
),
@@ -607,6 +609,10 @@ export class SettingsComponent
SETTINGS_KEYS.DEFAULT_PERMS_EDIT_GROUPS,
this.settingsForm.value.defaultPermsEditGroups
)
this.settings.set(
SETTINGS_KEYS.TAGS_SHOW_HIERARCHY,
this.settingsForm.value.tagsShowHierarchy
)
this.settings.set(
SETTINGS_KEYS.DOCUMENT_EDITING_REMOVE_INBOX_TAGS,
this.settingsForm.value.documentEditingRemoveInboxTags
@@ -186,7 +186,9 @@ export class AppFrameComponent
)
.subscribe((result) => {
this.documentList.reload(null, false)
this.documentList.reduceSelectionToFilter()
if (!this.documentList.allSelected) {
this.documentList.reduceSelectionToFilter()
}
// maybe someday we can optimize to only refresh affected documents
this.openDocumentsService.getOpenDocuments().forEach((doc) => {
this.openDocumentsService.refreshDocument(doc.id)
@@ -188,6 +188,62 @@ describe('ChatComponent', () => {
expect(component.loading()).toBe(false)
})
it('should reveal a complete answer within one second', () => {
component.input.set('Hello')
component.sendMessage()
const answer = 'A'.repeat(660)
mockStream$.next(answer)
mockStream$.complete()
expect(component.messages()[1].content.length).toBeGreaterThan(0)
expect(component.messages()[1].content.length).toBeLessThan(answer.length)
expect(component.loading()).toBe(true)
jest.advanceTimersByTime(800)
expect(component.messages()[1].content).toBe(answer)
expect(component.loading()).toBe(false)
})
it('should give a long answer more time to reveal', () => {
component.input.set('Hello')
component.sendMessage()
const answer = 'A'.repeat(3000)
mockStream$.next(answer)
mockStream$.complete()
jest.advanceTimersByTime(800)
expect(component.messages()[1].content.length).toBeLessThan(answer.length)
expect(component.loading()).toBe(true)
jest.advanceTimersByTime(2500)
expect(component.messages()[1].content).toBe(answer)
expect(component.loading()).toBe(false)
})
it('should reveal fast incoming chunks without making them pop in', () => {
component.input.set('Hello')
component.sendMessage()
let answer = ''
for (let i = 0; i < 20; i++) {
answer += 'A'.repeat(100)
const visibleBefore = component.messages()[1].content.length
mockStream$.next(answer)
jest.advanceTimersByTime(10)
expect(
component.messages()[1].content.length - visibleBefore
).toBeLessThan(100)
}
expect(component.messages()[1].content.length).toBeLessThan(answer.length)
mockStream$.complete()
jest.advanceTimersByTime(5200)
expect(component.messages()[1].content).toBe(answer)
expect(component.loading()).toBe(false)
})
it('should show a slow response message while waiting', () => {
component.input.set('Hello')
component.sendMessage()
@@ -21,6 +21,9 @@ import {
import { pngxPopperOptions } from 'src/app/utils/popper-options'
const SLOW_RESPONSE_DELAY_MS = 8000
const TYPEWRITER_MIN_DURATION_MS = 750
const TYPEWRITER_MAX_DURATION_MS = 3000
const TYPEWRITER_INTERVAL_MS = 16
@Component({
selector: 'pngx-chat',
@@ -73,6 +76,7 @@ export class ChatComponent implements OnInit {
private typewriterBuffer: string[] = []
private typewriterActive = false
private typewriterTimeout: ReturnType<typeof setTimeout>
private typewriterDeadline = 0
private streamSubscription: Subscription
private streamingMessage: ChatMessage
private streamFinished = false
@@ -134,7 +138,7 @@ export class ChatComponent implements OnInit {
if (nextResponse.content.length < lastVisibleContent.length) {
this.resetTypewriter(assistantMessage, nextResponse.content)
lastVisibleContent = nextResponse.content
} else {
} else if (nextResponse.content.length > lastVisibleContent.length) {
const visibleDelta = nextResponse.content.substring(
lastVisibleContent.length
)
@@ -214,6 +218,12 @@ export class ChatComponent implements OnInit {
if (!chunk) return
this.typewriterBuffer.push(...chunk.split(''))
// Longer bursts need more time to remain readable as the chat scrolls.
const revealDuration = Math.min(
TYPEWRITER_MAX_DURATION_MS,
Math.max(TYPEWRITER_MIN_DURATION_MS, this.typewriterBuffer.length)
)
this.typewriterDeadline = Date.now() + revealDuration
if (!this.typewriterActive) {
this.typewriterActive = true
@@ -222,19 +232,29 @@ export class ChatComponent implements OnInit {
}
playTypewriter(message: ChatMessage): void {
if (this.typewriterBuffer.length === 0) {
this.typewriterActive = false
if (this.streamFinished) this.finishStreaming(message)
return
}
const nextChar = this.typewriterBuffer.shift()
message.content += nextChar
// Spread the queued text across the updates left before the deadline
const remainingUpdates = Math.max(
1,
Math.ceil((this.typewriterDeadline - Date.now()) / TYPEWRITER_INTERVAL_MS)
)
const charactersToShow = Math.ceil(
this.typewriterBuffer.length / remainingUpdates
)
message.content += this.typewriterBuffer
.splice(0, charactersToShow)
.join('')
this.notifyMessagesChanged()
this.scrollToBottom()
// 10ms per character
this.typewriterTimeout = setTimeout(() => this.playTypewriter(message), 10)
if (this.typewriterBuffer.length === 0) {
this.typewriterActive = false
if (this.streamFinished) this.finishStreaming(message)
} else {
this.typewriterTimeout = setTimeout(
() => this.playTypewriter(message),
TYPEWRITER_INTERVAL_MS
)
}
}
private notifyMessagesChanged(): void {
@@ -0,0 +1,90 @@
import { provideHttpClient } from '@angular/common/http'
import { provideHttpClientTesting } from '@angular/common/http/testing'
import { Component } from '@angular/core'
import { ComponentFixture, TestBed } from '@angular/core/testing'
import { FormControl, ReactiveFormsModule } from '@angular/forms'
import { NumberComponent } from './number/number.component'
import { TextComponent } from './text/text.component'
import { TextAreaComponent } from './textarea/textarea.component'
import { UrlComponent } from './url/url.component'
@Component({
imports: [
ReactiveFormsModule,
TextComponent,
NumberComponent,
TextAreaComponent,
UrlComponent,
],
template: `
<pngx-input-text [formControl]="control" />
<pngx-input-number [formControl]="control" [showAdd]="false" />
<pngx-input-textarea [formControl]="control" />
<pngx-input-url [formControl]="control" />
`,
})
class InputEventsHostComponent {
control = new FormControl<string | number | null>(null)
}
describe.each([
['text', 'pngx-input-text input', 'Edited text', 'Edited text'],
['number', 'pngx-input-number input', '42', 42],
['textarea', 'pngx-input-textarea textarea', 'Edited\ntext', 'Edited\ntext'],
['url', 'pngx-input-url input', 'https://example.com', 'https://example.com'],
])(
'%s input events',
(_type, selector: string, text: string, value: string | number) => {
let fixture: ComponentFixture<InputEventsHostComponent>
let control: FormControl<string | number | null>
let input: HTMLInputElement | HTMLTextAreaElement
beforeEach(() => {
TestBed.configureTestingModule({
imports: [InputEventsHostComponent],
providers: [provideHttpClient(), provideHttpClientTesting()],
})
fixture = TestBed.createComponent(InputEventsHostComponent)
control = fixture.componentInstance.control
})
async function render() {
fixture.detectChanges()
await fixture.whenStable()
input = fixture.nativeElement.querySelector(selector)
}
it('updates the form while typing and marks it touched only on blur', async () => {
await render()
const changed = jest.fn()
control.valueChanges.subscribe(changed)
input.value = text
input.dispatchEvent(new Event('input'))
expect(control.value).toBe(value)
expect(control.dirty).toBe(true)
expect(control.touched).toBe(false)
input.dispatchEvent(new Event('change'))
input.dispatchEvent(new Event('blur'))
expect(control.touched).toBe(true)
expect(changed).toHaveBeenCalledTimes(1)
})
it('honors an explicit updateOn blur strategy', async () => {
control = new FormControl<string | number | null>(null, {
updateOn: 'blur',
})
fixture.componentInstance.control = control
await render()
input.value = text
input.dispatchEvent(new Event('input'))
expect(control.value).toBeNull()
expect(control.pristine).toBe(true)
input.dispatchEvent(new Event('blur'))
expect(control.value).toBe(value)
expect(control.dirty).toBe(true)
expect(control.touched).toBe(true)
})
}
)
@@ -12,7 +12,7 @@
</div>
<div class="position-relative" [class.col-md-9]="horizontal">
<div class="input-group" [class.is-invalid]="error">
<input #inputField type="number" class="form-control" [step]="step" [id]="inputId" [(ngModel)]="value" [ngModelOptions]="{standalone: true}" (change)="onChange(value)" [class.is-invalid]="error" [disabled]="disabled">
<input #inputField type="number" class="form-control" [step]="step" [id]="inputId" [(ngModel)]="value" [ngModelOptions]="{standalone: true}" (ngModelChange)="onChange($event)" (blur)="onTouched()" [class.is-invalid]="error" [disabled]="disabled">
@if (showAdd) {
<button class="btn btn-outline-secondary" type="button" id="button-addon1" (click)="nextAsn()" [disabled]="disabled">+1</button>
}
@@ -17,16 +17,21 @@
[searchFn]="searchFn"
addTagText="Add tag"
i18n-addTagText
[keyDownFn]="onKeyDown"
(add)="onAdd($event)"
(change)="onChange(value)">
<ng-template ng-label-tmp let-item="item">
<button class="tag-wrap btn p-0 d-flex align-items-center" (click)="removeTag(item.id)" (mousedown)="$event.stopImmediatePropagation()" type="button" title="Remove tag" i18n-title>
<pngx-bs-icon name="x" style="margin-inline-end: 1px;"></pngx-bs-icon>
@if (item.id && tags) {
<pngx-tag style="background-color: none;" [tag]="getTag(item.id)"></pngx-tag>
}
</button>
@if (isImplied(item.id)) {
<span class="implied-tag"></span>
} @else {
<button class="tag-wrap btn p-0 d-flex align-items-center mw-100" (click)="removeBadge(item.id)" (mousedown)="$event.stopImmediatePropagation()" type="button" title="Remove tag" i18n-title>
<pngx-bs-icon name="x" style="margin-inline-end: 1px;"></pngx-bs-icon>
@if (item.id && tags) {
<pngx-tag class="mw-100 overflow-hidden" [tag]="getTag(item.id)" [split]="preserveHierarchy" (click)="removeTag(item.id); $event.stopPropagation()" (parentClick)="removeTag($event)"></pngx-tag>
}
</button>
}
</ng-template>
<ng-template ng-option-tmp let-item="item" let-index="index" let-search="searchTerm">
<div class="tag-option-row d-flex align-items-center" [class.w-auto]="!getTag(item.id)?.parent">
@@ -7,6 +7,11 @@
font-size: 1rem;
}
// ancestors of another selected tag render inside that tag's badge
::ng-deep .paperless-input-tags .ng-value:has(> .implied-tag) {
display: none !important;
}
.paperless-input-select.disabled {
.input-group {
cursor: not-allowed;
@@ -278,4 +278,59 @@ describe('TagsComponent', () => {
component.tags = [lone]
expect(component.getParentChain(5)).toEqual([])
})
describe('hierarchy', () => {
const root: Tag = { id: 1, name: 'root' }
const mid: Tag = { id: 2, name: 'mid', parent: 1 }
const leaf: Tag = { id: 3, name: 'leaf', parent: 2 }
const other: Tag = { id: 4, name: 'other' }
beforeEach(() => {
component.tags = [root, mid, leaf, other]
component.value = [3, 2, 1, 4]
})
it('should treat ancestors of another selected tag as implied', () => {
expect([1, 2, 3, 4].map((id) => component.isImplied(id))).toEqual([
true,
true,
false,
false,
])
component.preserveHierarchy = false
expect(component.isImplied(1)).toBe(false)
})
it('should remove a whole badge, keeping ancestors other tags still need', () => {
component.removeBadge(3)
expect(component.value).toEqual([4])
const sibling: Tag = { id: 5, name: 'sibling', parent: 1 }
component.tags = [root, mid, leaf, other, sibling]
component.value = [3, 2, 1, 5, 4]
component.removeBadge(3)
expect(component.value).toEqual([1, 5, 4])
})
it('should remove only the tag itself from a badge when hierarchy is not shown', () => {
component.preserveHierarchy = false
component.removeBadge(3)
expect(component.value).toEqual([2, 1, 4])
})
it('should remove the last visible tag on backspace, not a hidden ancestor', () => {
const backspace = new KeyboardEvent('keydown', { key: 'Backspace' })
expect(component.onKeyDown(backspace)).toBe(false)
expect(component.value).toEqual([3, 2, 1])
expect(component.onKeyDown(backspace)).toBe(false)
expect(component.value).toEqual([2, 1])
// nothing hidden: leave it to ng-select
component.value = [4]
expect(component.onKeyDown(backspace)).toBe(true)
expect(
component.onKeyDown(new KeyboardEvent('keydown', { key: 'a' }))
).toBe(true)
})
})
})
@@ -21,7 +21,13 @@ import { NgSelectComponent, NgSelectModule } from '@ng-select/ng-select'
import { first, firstValueFrom, tap } from 'rxjs'
import { BootstrapIconComponent } from 'src/app/components/common/bootstrap-icon/bootstrap-icon.component'
import { Tag } from 'src/app/data/tag'
import { SETTINGS_KEYS } from 'src/app/data/ui-settings'
import { TagService } from 'src/app/services/rest/tag.service'
import { SettingsService } from 'src/app/services/settings.service'
import {
collapseImpliedAncestors,
getTagAncestors,
} from 'src/app/utils/collapse-tags'
import { matchesSearchText } from 'src/app/utils/text-search'
import { EditDialogMode } from '../../edit-dialog/edit-dialog.component'
import { TagEditDialogComponent } from '../../edit-dialog/tag-edit-dialog/tag-edit-dialog.component'
@@ -50,6 +56,7 @@ import { TagComponent } from '../../tag/tag.component'
export class TagsComponent implements OnInit, ControlValueAccessor {
private tagService = inject(TagService)
private modalService = inject(NgbModal)
private readonly settingsService = inject(SettingsService)
private readonly changeDetector = inject(ChangeDetectorRef)
constructor() {
@@ -131,6 +138,33 @@ export class TagsComponent implements OnInit, ControlValueAccessor {
term
)
/**
* Selected tags that are ancestors of another selected tag which are shown
* as segments of their descendant's badge.
*/
isImplied(id: number): boolean {
if (!this.showsHierarchy || !this.tags || !this.value) return false
return !collapseImpliedAncestors(this.value, this.tagsById).includes(id)
}
private get showsHierarchy(): boolean {
return (
this.preserveHierarchy &&
!!this.settingsService.get(SETTINGS_KEYS.TAGS_SHOW_HIERARCHY)
)
}
/**
* Remove the last visible tag (in case its actually a hidden ancestor)
*/
readonly onKeyDown = (event: KeyboardEvent): boolean => {
if (event.key !== 'Backspace' || this.select?.searchTerm) return true
const visible = this.value?.filter((id) => !this.isImplied(id)) ?? []
if (visible.length === (this.value?.length ?? 0)) return true
this.removeTag(visible.at(-1))
return false
}
getTag(id: number) {
if (this.tags) {
return this.tags.find((tag) => tag.id == id)
@@ -139,6 +173,26 @@ export class TagsComponent implements OnInit, ControlValueAccessor {
}
}
/**
* Remove a whole badge i.e. the tag plus any ancestors that no other selected
* tag still needs.
*/
removeBadge(tagID: number) {
if (this.disabled) return
if (!this.showsHierarchy) return this.removeTag(tagID)
const ancestors = new Set(this.getParentChain(tagID).map((t) => t.id))
const needed = new Set(
this.value
.filter((id) => id !== tagID && !ancestors.has(id))
.flatMap((id) => this.getParentChain(id).map((t) => t.id))
)
this.value = this.value.filter(
(id) => id !== tagID && (!ancestors.has(id) || needed.has(id))
)
this.onChange(this.value)
}
removeTag(tagID: number) {
if (this.disabled) return
@@ -235,18 +289,17 @@ export class TagsComponent implements OnInit, ControlValueAccessor {
}
getParentChain(id: number): Tag[] {
// Returns ancestors from root → immediate parent for a tag id
const chain: Tag[] = []
let current = this.getTag(id)
const guard = new Set<number>()
while (current?.parent) {
if (guard.has(current.parent)) break
guard.add(current.parent)
const parent = this.getTag(current.parent)
if (!parent) break
chain.unshift(parent)
current = parent
return getTagAncestors(id, this.tagsById)
}
private tagsByIdSource: Tag[]
private tagsByIdCache: Map<number, Tag>
private get tagsById(): Map<number, Tag> {
if (this.tagsByIdSource !== this.tags) {
this.tagsByIdSource = this.tags
this.tagsByIdCache = new Map(this.tags?.map((t) => [t.id, t]))
}
return chain
return this.tagsByIdCache
}
}
@@ -11,7 +11,7 @@
}
</div>
<div class="position-relative" [class.col-md-9]="horizontal">
<input #inputField type="text" class="form-control" [class.is-invalid]="error" [id]="inputId" [(ngModel)]="value" [ngModelOptions]="{standalone: true}" (change)="onChange(value)" [disabled]="disabled" [autocomplete]="autocomplete" [placeholder]="placeholder">
<input #inputField type="text" class="form-control" [class.is-invalid]="error" [id]="inputId" [(ngModel)]="value" [ngModelOptions]="{standalone: true}" (ngModelChange)="onChange($event)" (blur)="onTouched()" [disabled]="disabled" [autocomplete]="autocomplete" [placeholder]="placeholder">
@if (hint) {
<small class="form-text text-muted" [innerHTML]="hint"></small>
}
@@ -18,7 +18,8 @@
[class.font-monospace]="monospace"
[(ngModel)]="value"
[ngModelOptions]="{standalone: true}"
(change)="onChange(value)"
(ngModelChange)="onChange($event)"
(blur)="onTouched()"
[disabled]="disabled"
[placeholder]="placeholder"
rows="4">
@@ -10,7 +10,7 @@
</div>
<div [class.col-md-9]="horizontal">
<div class="input-group" [class.is-invalid]="error">
<input #inputField type="url" class="form-control" [class.is-invalid]="error" placeholder="https://" [id]="inputId" [(ngModel)]="value" [ngModelOptions]="{standalone: true}" (change)="onChange(value)" [disabled]="disabled">
<input #inputField type="url" class="form-control" [class.is-invalid]="error" placeholder="https://" [id]="inputId" [(ngModel)]="value" [ngModelOptions]="{standalone: true}" (ngModelChange)="onChange($event)" (blur)="onTouched()" [disabled]="disabled">
<a class="btn btn-outline-secondary rounded-end" title="Open link" i18n-title [href]="value" target="_blank">
<pngx-bs-icon width="1.2em" height="1.2em" name="box-arrow-up-right"></pngx-bs-icon>
</a>
@@ -1,13 +1,18 @@
@if (tag()) {
@if (showParents() && tag().parent) {
<pngx-tag [tagID]="tag().parent" [clickable]="clickable()" [linkTitle]="linkTitle()"></pngx-tag>
&nbsp;&gt;&nbsp;
}
@if (!clickable()) {
<span class="badge" [style.background]="tag().color" [style.color]="tag().text_color">{{tag().name}}</span>
}
@if (clickable()) {
<a [title]="linkTitle()" class="badge" [style.background]="tag().color" [style.color]="tag().text_color">{{tag().name}}</a>
@if (ancestorSegments.length) {
<span #pill class="split-pill d-inline-flex mw-100" [title]="path()" (mouseleave)="setFocus(pill, null)">
@for (ancestor of ancestorSegments; track ancestor.id; let first = $first) {
<span #segment class="badge split-parent" [class.rounded-end-0]="first" [class.rounded-0]="!first" [style.background]="ancestor.color" [style.color]="ancestor.text_color" (mouseenter)="setFocus(pill, segment)" (click)="onParentClick($event, ancestor.id)">{{ancestor.name}}</span>
}
<span #leaf class="badge rounded-start-0 split-child" [style.background]="tag().color" [style.color]="tag().text_color" (mouseenter)="setFocus(pill, leaf)">{{tag().name}}</span>
</span>
} @else {
@if (!clickable()) {
<span class="badge" [style.background]="tag().color" [style.color]="tag().text_color">{{tag().name}}</span>
}
@if (clickable()) {
<a [title]="linkTitle()" class="badge" [style.background]="tag().color" [style.color]="tag().text_color">{{tag().name}}</a>
}
}
} @else if (loading) {
<span class="placeholder-glow">
@@ -11,3 +11,45 @@ a, span {
opacity: .5;
font-style: italic;
}
.split-parent,
.split-child {
overflow: hidden;
white-space: nowrap;
transition: color .2s ease;
}
.split-parent {
min-width: 1.5em;
max-width: 10em;
padding-inline: .4em;
text-overflow: clip;
&:nth-last-child(2) { flex-shrink: 1e12; }
&:nth-last-child(3) { flex-shrink: 1e18; }
&:nth-last-child(4) { flex-shrink: 1e24; }
&:nth-last-child(5) { flex-shrink: 1e30; }
&.collapsed {
color: transparent !important;
}
}
.split-child {
flex-shrink: 1e6;
min-width: 2em;
text-overflow: ellipsis;
}
// Hovered segment gets priority, other parents can shrink to a sliver
.split-pill {
> .split-parent.focused,
> .split-child.focused {
flex-shrink: 1;
text-overflow: ellipsis;
}
&:has(> .focused) > .split-parent:not(.focused) {
min-width: 0;
}
}
@@ -6,6 +6,7 @@ import { of } from 'rxjs'
import { Tag } from 'src/app/data/tag'
import { PermissionsService } from 'src/app/services/permissions.service'
import { TagService } from 'src/app/services/rest/tag.service'
import { SettingsService } from 'src/app/services/settings.service'
import { TagComponent } from './tag.component'
const tag: Tag = {
@@ -69,4 +70,88 @@ describe('TagComponent', () => {
expect(getCachedSpy).toHaveBeenCalledWith(1)
expect(component.tag()).toEqual(tag)
})
describe('split badge', () => {
const root: Tag = { id: 1, name: 'Root', color: '#000000' }
const mid: Tag = { id: 2, name: 'Mid', color: '#111111', parent: 1 }
const leaf: Tag = { id: 3, name: 'Leaf', color: '#222222', parent: 2 }
const segments = (): HTMLElement[] =>
fixture.debugElement
.queryAll(By.css('.split-pill > .badge'))
.map((d) => d.nativeElement)
beforeEach(() => {
jest.spyOn(permissionsService, 'currentUserCan').mockReturnValue(true)
jest.spyOn(tagService, 'getCached').mockReturnValue(of(leaf))
jest
.spyOn(tagService, 'listAll')
.mockReturnValue(
of({ count: 3, all: [1, 2, 3], results: [root, mid, leaf] })
)
fixture.componentRef.setInput('tagID', 3)
fixture.componentRef.setInput('split', true)
fixture.detectChanges()
})
it('should render ancestors as segments with the full path as title', () => {
expect(segments().map((s) => s.textContent.trim())).toEqual([
'Root',
'Mid',
'Leaf',
])
expect(
fixture.debugElement.query(By.css('.split-pill')).nativeElement.title
).toEqual('Root › Mid › Leaf')
})
it('should render a plain badge when the setting is off', () => {
jest.spyOn(TestBed.inject(SettingsService), 'get').mockReturnValue(false)
fixture.componentRef.setInput('split', false)
fixture.detectChanges()
fixture.componentRef.setInput('split', true)
fixture.detectChanges()
expect(fixture.debugElement.query(By.css('.split-pill'))).toBeNull()
expect(component.ancestorSegments).toEqual([])
})
it('should emit the ancestor id when an ancestor segment is clicked', () => {
const emitSpy = jest.spyOn(component.parentClick, 'emit')
const event = new MouseEvent('click', { bubbles: true })
const stopSpy = jest.spyOn(event, 'stopImmediatePropagation')
segments()[1].dispatchEvent(event)
expect(emitSpy).toHaveBeenCalledWith(2)
expect(stopSpy).toHaveBeenCalled()
})
it('should focus the hovered segment and animate width changes', () => {
const [rootSeg, midSeg, leafSeg] = segments()
const pill = rootSeg.parentElement
// before / after widths per segment, as measured by setFocus
const widths = new Map<HTMLElement, number[]>([
[rootSeg, [20, 4]],
[midSeg, [20, 60]],
[leafSeg, [40, 40]],
])
const animate = jest.fn()
for (const [seg, [before, after]] of widths) {
seg.getAnimations = jest.fn(() => [])
seg.animate = animate
jest
.spyOn(seg, 'getBoundingClientRect')
.mockReturnValueOnce({ width: before } as DOMRect)
.mockReturnValueOnce({ width: after } as DOMRect)
}
midSeg.dispatchEvent(new MouseEvent('mouseenter'))
expect(midSeg.classList).toContain('focused')
expect(rootSeg.classList).not.toContain('focused')
// only the segments whose width changed are animated
expect(animate).toHaveBeenCalledTimes(2)
pill.dispatchEvent(new MouseEvent('mouseleave'))
expect(segments().some((s) => s.classList.contains('focused'))).toBe(
false
)
})
})
})
@@ -1,11 +1,27 @@
import { Component, effect, inject, input, model } from '@angular/core'
import {
Component,
computed,
effect,
inject,
input,
model,
output,
signal,
} from '@angular/core'
import { Tag } from 'src/app/data/tag'
import { SETTINGS_KEYS } from 'src/app/data/ui-settings'
import {
PermissionAction,
PermissionsService,
PermissionType,
} from 'src/app/services/permissions.service'
import { TagService } from 'src/app/services/rest/tag.service'
import { SettingsService } from 'src/app/services/settings.service'
import { getTagAncestors } from 'src/app/utils/collapse-tags'
const FOCUS_ANIMATION_MS = 200
// matches the .split-parent min-width in tag.component.scss
const COLLAPSE_BELOW_EM = 1.5
@Component({
selector: 'pngx-tag',
@@ -13,8 +29,9 @@ import { TagService } from 'src/app/services/rest/tag.service'
styleUrls: ['./tag.component.scss'],
})
export class TagComponent {
private permissionsService = inject(PermissionsService)
private tagService = inject(TagService)
private readonly permissionsService = inject(PermissionsService)
private readonly tagService = inject(TagService)
private readonly settingsService = inject(SettingsService)
readonly tag = model<Tag>(null)
readonly tagID = input<number>(undefined)
@@ -23,27 +40,101 @@ export class TagComponent {
transform: (value: boolean | string) =>
value === '' || value === true || value === 'true',
})
readonly showParents = input(false)
readonly split = input(false)
readonly parentClick = output<number>()
private readonly ancestors = signal<Tag[]>([])
public get ancestorSegments(): Tag[] {
return this.split() &&
this.settingsService.get(SETTINGS_KEYS.TAGS_SHOW_HIERARCHY)
? this.ancestors()
: []
}
public readonly path = computed(() =>
[...this.ancestors(), this.tag()]
.filter(Boolean)
.map((t) => t.name)
.join(' › ')
)
constructor() {
effect(() => {
const tagID = this.tagID()
if (tagID) {
if (
this.permissionsService.currentUserCan(
PermissionAction.View,
PermissionType.Tag
)
) {
this.tagService.getCached(tagID).subscribe((tag) => {
this.tag.set(tag)
})
}
if (tagID && this.canViewTags) {
this.tagService.getCached(tagID).subscribe((tag) => {
this.tag.set(tag)
})
}
})
// ancestors for split badges, from either tagID or a passed-in tag
effect(() => {
const id = this.tagID() || this.tag()?.id
if (!this.split() || !id || !this.canViewTags) {
this.ancestors.set([])
return
}
this.tagService.listAll().subscribe((tags) => {
this.ancestors.set(
getTagAncestors(
id,
new Map<number, Tag>(tags.results.map((t) => [t.id, t]))
)
)
})
})
}
private get canViewTags(): boolean {
return this.permissionsService.currentUserCan(
PermissionAction.View,
PermissionType.Tag
)
}
public get loading(): boolean {
return this.tagService.loading
}
/**
* Gives the hovered segment priority (.focused class, see scss)
* and animates each from old width to new
*/
setFocus(segment: HTMLElement, focused: HTMLElement | null) {
const segments = Array.from(segment.children) as HTMLElement[]
const before = segments.map((s) => s.getBoundingClientRect().width)
for (const segment of segments) {
segment.getAnimations().forEach((a) => a.cancel())
segment.classList.toggle('focused', segment === focused)
}
const after = segments.map((s) => s.getBoundingClientRect().width)
segments.forEach((segment, i) => {
const em = Number.parseFloat(getComputedStyle(segment).fontSize)
segment.classList.toggle(
'collapsed',
!!focused &&
segment.classList.contains('split-parent') &&
after[i] < COLLAPSE_BELOW_EM * em
)
if (Math.abs(after[i] - before[i]) < 0.5) return
segment.animate(
[
{ flex: `0 0 ${before[i]}px`, minWidth: 0 },
{ flex: `0 0 ${after[i]}px`, minWidth: 0 },
],
{ duration: FOCUS_ANIMATION_MS, easing: 'ease' }
)
})
}
onParentClick(event: MouseEvent, id: number) {
event.preventDefault()
event.stopImmediatePropagation()
this.parentClick.emit(id)
}
}
@@ -76,8 +76,8 @@
}
}
@case (DisplayField.TAGS) {
@for (tagID of doc.tags; track tagID) {
<pngx-tag [tagID]="tagID" class="ms-1 fs-6" (click)="clickTag(tagID, $event)" [clickable]="true" linkTitle="Filter by tag" i18n-title></pngx-tag>
@for (tagID of doc.tags | collapseTags | async; track tagID) {
<pngx-tag [tagID]="tagID" [split]="true" class="ms-1 fs-6" (click)="clickTag(tagID, $event)" (parentClick)="clickTag($event)" [clickable]="true" linkTitle="Filter by tag" i18n-title></pngx-tag>
}
}
@case (DisplayField.DOCUMENT_TYPE) {
@@ -38,6 +38,7 @@ import {
} from 'src/app/data/filter-rule-type'
import { SavedView } from 'src/app/data/saved-view'
import { IfPermissionsDirective } from 'src/app/directives/if-permissions.directive'
import { CollapseTagsPipe } from 'src/app/pipes/collapse-tags.pipe'
import { CorrespondentNamePipe } from 'src/app/pipes/correspondent-name.pipe'
import { CustomDatePipe } from 'src/app/pipes/custom-date.pipe'
import { DocumentTitlePipe } from 'src/app/pipes/document-title.pipe'
@@ -75,6 +76,7 @@ import { WidgetFrameComponent } from '../widget-frame/widget-frame.component'
DocumentTypeNamePipe,
StoragePathNamePipe,
AsyncPipe,
CollapseTagsPipe,
DocumentTitlePipe,
CustomDatePipe,
NgClass,
@@ -390,14 +390,12 @@ describe('DocumentDetailComponent', () => {
expect(component.documentForm.get('title').value).toEqual('Foo Bar')
})
it('should update title before doc change if was not updated via debounce', () => {
it('should update the title form value without waiting for debounce or blur', () => {
jest.useFakeTimers()
initNormally()
component.titleInput.value = 'Foo Bar'
component.titleInput.inputField.nativeElement.dispatchEvent(
new Event('change')
)
jest.advanceTimersByTime(1000)
const input = component.titleInput.inputField.nativeElement
input.value = 'Foo Bar'
input.dispatchEvent(new Event('input'))
expect(component.documentForm.get('title').value).toEqual('Foo Bar')
})
@@ -718,7 +718,6 @@ describe('BulkEditorComponent', () => {
'Workflow will run on the selected documents in the background.'
)
expect(toast.mock.calls.at(-1)[0].actionName).toBeUndefined()
expect(documentListViewService.selected.size).toBe(0)
httpTestingController.match(
`${environment.apiBaseUrl}documents/?page=1&page_size=50&ordering=-created&truncate_content=true&include_selection_data=true`
) // list reload
@@ -1293,7 +1292,7 @@ describe('BulkEditorComponent', () => {
)
req.flush({ result: 'OK', task_id: 'delete-2' })
// ids aren't known when all matching documents are selected, so no undo
expect(trackSpy).toHaveBeenLastCalledWith('delete-2', 0, null)
expect(trackSpy).toHaveBeenLastCalledWith('delete-2', 2, null)
httpTestingController.match(
`${environment.apiBaseUrl}documents/?page=1&page_size=50&ordering=-created&truncate_content=true&include_selection_data=true`
) // list reload
@@ -1492,7 +1491,6 @@ describe('BulkEditorComponent', () => {
httpTestingController.match(
`${environment.apiBaseUrl}documents/?page=1&page_size=100000&fields=id`
) // listAllFilteredIds
expect(documentListViewService.selected.size).toEqual(0)
// Test with archiveFallback enabled
modal.componentInstance.deleteOriginals.set(false)
@@ -1514,7 +1512,6 @@ describe('BulkEditorComponent', () => {
httpTestingController.match(
`${environment.apiBaseUrl}documents/?page=1&page_size=100000&fields=id`
) // listAllFilteredIds
expect(documentListViewService.selected.size).toEqual(0)
})
it('should support merging documents as versions', () => {
@@ -1562,7 +1559,6 @@ describe('BulkEditorComponent', () => {
httpTestingController.match(
`${environment.apiBaseUrl}documents/?page=1&page_size=100000&fields=id`
)
expect(documentListViewService.selected.size).toEqual(0)
jest.advanceTimersByTime(400)
expect(toastSpy).toHaveBeenCalledWith(
expect.objectContaining({
@@ -1717,7 +1713,6 @@ describe('BulkEditorComponent', () => {
expect(notification.actionName).toBeUndefined()
}
expect(reload).not.toHaveBeenCalled()
expect(documentListViewService.selected.size).toBe(0)
expect(documentListViewService.allSelected).toBe(false)
TestBed.inject(TaskNotificationService).complete({
task_id: 'queued-operation',
@@ -329,7 +329,6 @@ export class BulkEditorComponent
.subscribe({
next: () => {
modal.close()
this.list.selectNone()
this.toastService.showInfo(
$localize`Workflow will run on the selected documents in the background.`
)
@@ -366,7 +365,6 @@ export class BulkEditorComponent
) {
const bulk = this.getSelectionSize() > 1
modal?.close()
this.list.selectNone()
if (onQueued) {
onQueued(taskId)
} else {
@@ -36,8 +36,8 @@
{{document().title | documentTitle}}
}
@if (displayFields().includes(DisplayField.TAGS)) {
@for (tagID of document().tags; track tagID) {
<pngx-tag [tagID]="tagID" linkTitle="Filter by tag" i18n-linkTitle class="ms-1" (click)="clickTag.emit(tagID);$event.stopPropagation()" [clickable]="clickTag.observers.length"></pngx-tag>
@for (tagID of document().tags | collapseTags | async; track tagID) {
<pngx-tag [tagID]="tagID" [split]="true" linkTitle="Filter by tag" i18n-linkTitle class="ms-1" (click)="clickTag.emit(tagID);$event.stopPropagation()" (parentClick)="clickTag.emit($event)" [clickable]="clickTag.observers.length"></pngx-tag>
}
}
} @else {
@@ -20,6 +20,7 @@ import {
Document,
} from 'src/app/data/document'
import { SETTINGS_KEYS } from 'src/app/data/ui-settings'
import { CollapseTagsPipe } from 'src/app/pipes/collapse-tags.pipe'
import { CorrespondentNamePipe } from 'src/app/pipes/correspondent-name.pipe'
import { CustomDatePipe } from 'src/app/pipes/custom-date.pipe'
import { DocumentTitlePipe } from 'src/app/pipes/document-title.pipe'
@@ -45,6 +46,7 @@ import { LoadingComponentWithPermissions } from '../../loading-component/loading
TagComponent,
CustomFieldDisplayComponent,
AsyncPipe,
CollapseTagsPipe,
NgOptimizedImage,
UsernamePipe,
CorrespondentNamePipe,
@@ -19,11 +19,12 @@
}
@if (document() && displayFields().includes(DisplayField.TAGS)) {
<div class="tags d-flex flex-column text-end position-absolute me-1 fs-6" [class.tags-no-wrap]="document().tags.length > 3">
@for (tagID of tagIDs; track tagID) {
<pngx-tag [tagID]="tagID" (click)="clickTag.emit(tagID);$event.stopPropagation()" [clickable]="true" linkTitle="Toggle tag filter" i18n-linkTitle></pngx-tag>
@let visibleTagIDs = document().tags | collapseTags | async;
<div class="tags d-flex flex-column text-end position-absolute me-1 fs-6" [class.tags-no-wrap]="visibleTagIDs?.length > 3">
@for (tagID of getTagIDs(visibleTagIDs); track tagID) {
<pngx-tag [tagID]="tagID" [split]="true" (click)="clickTag.emit(tagID);$event.stopPropagation()" (parentClick)="clickTag.emit($event)" [clickable]="true" linkTitle="Toggle tag filter" i18n-linkTitle></pngx-tag>
}
@if (moreTags) {
@if (getMoreTags(visibleTagIDs); as moreTags) {
<div>
<span class="badge text-dark">+ {{moreTags}}</span>
</div>
@@ -103,7 +103,8 @@ a {
line-height: 1;
&.tags-no-wrap {
::ng-deep .badge {
// split-pill parents clip to a stub instead (see TagComponent)
::ng-deep .badge:not(.split-parent) {
display: inline-block;
max-width: 100%;
white-space: nowrap;
@@ -4,6 +4,9 @@ import { provideHttpClientTesting } from '@angular/common/http/testing'
import { ComponentFixture, TestBed } from '@angular/core/testing'
import { By } from '@angular/platform-browser'
import { RouterTestingModule } from '@angular/router/testing'
import { of } from 'rxjs'
import { PermissionsService } from 'src/app/services/permissions.service'
import { TagService } from 'src/app/services/rest/tag.service'
import { TagComponent } from '../../common/tag/tag.component'
import { DocumentCardSmallComponent } from './document-card-small.component'
@@ -97,8 +100,36 @@ describe('DocumentCardSmallComponent', () => {
).toHaveLength(6)
})
it('should hide parent tags implied by a child and count pills for the hidden tag counter', () => {
jest
.spyOn(TestBed.inject(PermissionsService), 'currentUserCan')
.mockReturnValue(true)
jest.spyOn(TestBed.inject(TagService), 'listAll').mockReturnValue(
of({
count: 8,
all: [1, 2, 3, 4, 5, 6, 7, 8],
results: [
{ id: 1, name: 'Parent' },
{ id: 2, name: 'Child', parent: 1 },
...[3, 4, 5, 6, 7, 8].map((id) => ({ id, name: `Tag ${id}` })),
],
})
)
fixture.componentRef.setInput('document', {
...component.document(),
tags: [1, 2, 3, 4, 5, 6, 7, 8],
})
fixture.detectChanges()
const tags = fixture.debugElement.queryAll(By.directive(TagComponent))
expect(tags.map((t) => t.componentInstance.tagID())).toEqual([
2, 3, 4, 5, 6,
])
expect(fixture.nativeElement.textContent).toContain('+ 2')
})
it('should clear hidden tag counter when tag count falls below the limit', () => {
expect(component.moreTags).toEqual(3)
expect(component.getMoreTags(component.document().tags)).toEqual(3)
fixture.componentRef.setInput('document', {
...component.document(),
@@ -106,7 +137,7 @@ describe('DocumentCardSmallComponent', () => {
})
fixture.detectChanges()
expect(component.moreTags).toBeNull()
expect(component.getMoreTags(component.document().tags)).toBeNull()
expect(fixture.nativeElement.textContent).not.toContain('+ 3')
})
@@ -20,6 +20,7 @@ import {
Document,
} from 'src/app/data/document'
import { SETTINGS_KEYS } from 'src/app/data/ui-settings'
import { CollapseTagsPipe } from 'src/app/pipes/collapse-tags.pipe'
import { CorrespondentNamePipe } from 'src/app/pipes/correspondent-name.pipe'
import { CustomDatePipe } from 'src/app/pipes/custom-date.pipe'
import { DocumentTitlePipe } from 'src/app/pipes/document-title.pipe'
@@ -45,6 +46,7 @@ import { LoadingComponentWithPermissions } from '../../loading-component/loading
TagComponent,
CustomFieldDisplayComponent,
AsyncPipe,
CollapseTagsPipe,
NgOptimizedImage,
UsernamePipe,
CorrespondentNamePipe,
@@ -90,12 +92,13 @@ export class DocumentCardSmallComponent
@Output()
clickStoragePath = new EventEmitter<number>()
get moreTags(): number {
const document = this.document()
const limit = document?.notes.length > 0 ? 6 : 7
return document?.tags.length > limit
? document.tags.length - (limit - 1)
: null
private get tagLimit(): number {
return this.document()?.notes.length > 0 ? 6 : 7
}
getMoreTags(tagIDs: number[]): number {
const limit = this.tagLimit
return tagIDs?.length > limit ? tagIDs.length - (limit - 1) : null
}
@ViewChild('popupPreview') popupPreview: PreviewPopupComponent
@@ -116,14 +119,9 @@ export class DocumentCardSmallComponent
return this.documentService.getDownloadUrl(this.document().id)
}
get tagIDs() {
const document = this.document()
const limit = document.notes.length > 0 ? 6 : 7
if (document.tags.length > limit) {
return document.tags.slice(0, limit - 1)
} else {
return document.tags
}
getTagIDs(tagIDs: number[]): number[] {
const limit = this.tagLimit
return tagIDs?.length > limit ? tagIDs.slice(0, limit - 1) : tagIDs
}
mouseLeaveCard() {
@@ -335,8 +335,8 @@
<a routerLink="/documents/{{d.id}}" title="Edit document" i18n-title style="overflow-wrap: anywhere;">{{d.title | documentTitle}}</a>
}
@if (activeDisplayFields.includes(DisplayField.TAGS)) {
@for (tagID of d.tags; track tagID) {
<pngx-tag [tagID]="tagID" class="ms-1 fs-6" clickable="true" linkTitle="Filter by tag" i18n-linkTitle (click)="clickTag(tagID);$event.stopPropagation()"></pngx-tag>
@for (tagID of d.tags | collapseTags | async; track tagID) {
<pngx-tag [tagID]="tagID" [split]="true" class="ms-1 fs-6" clickable="true" linkTitle="Filter by tag" i18n-linkTitle (click)="clickTag(tagID);$event.stopPropagation()" (parentClick)="clickTag($event)"></pngx-tag>
}
}
</td>
@@ -39,6 +39,7 @@ import {
SortableDirective,
SortEvent,
} from 'src/app/directives/sortable.directive'
import { CollapseTagsPipe } from 'src/app/pipes/collapse-tags.pipe'
import { CorrespondentNamePipe } from 'src/app/pipes/correspondent-name.pipe'
import { CustomDatePipe } from 'src/app/pipes/custom-date.pipe'
import { DocumentTitlePipe } from 'src/app/pipes/document-title.pipe'
@@ -96,6 +97,7 @@ import { SaveViewConfigDialogComponent } from './save-view-config-dialog/save-vi
StoragePathNamePipe,
BootstrapIconComponent,
AsyncPipe,
CollapseTagsPipe,
FormsModule,
ReactiveFormsModule,
NgTemplateOutlet,
+6
View File
@@ -94,6 +94,7 @@ export const SETTINGS_KEYS = {
DEFAULT_PERMS_VIEW_GROUPS: 'general-settings:permissions:default-view-groups',
DEFAULT_PERMS_EDIT_USERS: 'general-settings:permissions:default-edit-users',
DEFAULT_PERMS_EDIT_GROUPS: 'general-settings:permissions:default-edit-groups',
TAGS_SHOW_HIERARCHY: 'general-settings:tags:show-hierarchy',
DOCUMENT_EDITING_REMOVE_INBOX_TAGS:
'general-settings:document-editing:remove-inbox-tags',
DOCUMENT_EDITING_OVERLAY_THUMBNAIL:
@@ -305,6 +306,11 @@ export const SETTINGS: UiSetting[] = [
type: 'string',
default: '',
},
{
key: SETTINGS_KEYS.TAGS_SHOW_HIERARCHY,
type: 'boolean',
default: true,
},
{
key: SETTINGS_KEYS.DOCUMENT_EDITING_REMOVE_INBOX_TAGS,
type: 'boolean',
@@ -0,0 +1,67 @@
import { provideHttpClient, withInterceptorsFromDi } from '@angular/common/http'
import { provideHttpClientTesting } from '@angular/common/http/testing'
import { TestBed } from '@angular/core/testing'
import { firstValueFrom, of, throwError } from 'rxjs'
import { Tag } from '../data/tag'
import { PermissionsService } from '../services/permissions.service'
import { TagService } from '../services/rest/tag.service'
import { SettingsService } from '../services/settings.service'
import { CollapseTagsPipe } from './collapse-tags.pipe'
const tags: Tag[] = [
{ id: 1, name: 'Parent' },
{ id: 2, name: 'Child', parent: 1 },
{ id: 3, name: 'Other' },
]
describe('CollapseTagsPipe', () => {
let pipe: CollapseTagsPipe
let tagService: TagService
let settingsService: SettingsService
let permissionsService: PermissionsService
beforeEach(() => {
TestBed.configureTestingModule({
providers: [
CollapseTagsPipe,
provideHttpClient(withInterceptorsFromDi()),
provideHttpClientTesting(),
],
})
pipe = TestBed.inject(CollapseTagsPipe)
tagService = TestBed.inject(TagService)
settingsService = TestBed.inject(SettingsService)
permissionsService = TestBed.inject(PermissionsService)
jest.spyOn(permissionsService, 'currentUserCan').mockReturnValue(true)
jest.spyOn(settingsService, 'get').mockReturnValue(true)
jest
.spyOn(tagService, 'listAll')
.mockReturnValue(of({ count: 3, all: [1, 2, 3], results: tags }))
})
it('should drop parents implied by a selected child', async () => {
expect(await firstValueFrom(pipe.transform([1, 2, 3]))).toEqual([2, 3])
})
it('should pass ids through when hierarchy display is off or not permitted', async () => {
jest.spyOn(settingsService, 'get').mockReturnValue(false)
expect(await firstValueFrom(pipe.transform([1, 2]))).toEqual([1, 2])
jest.spyOn(settingsService, 'get').mockReturnValue(true)
jest.spyOn(permissionsService, 'currentUserCan').mockReturnValue(false)
expect(await firstValueFrom(pipe.transform([1, 2]))).toEqual([1, 2])
expect(tagService.listAll).not.toHaveBeenCalled()
})
it('should handle empty input', async () => {
expect(await firstValueFrom(pipe.transform(null))).toEqual([])
expect(await firstValueFrom(pipe.transform([]))).toEqual([])
})
it('should fall back to the original ids if tags fail to load', async () => {
jest
.spyOn(tagService, 'listAll')
.mockReturnValue(throwError(() => new Error('failed')))
expect(await firstValueFrom(pipe.transform([1, 2]))).toEqual([1, 2])
})
})
@@ -0,0 +1,47 @@
import { inject, Pipe, PipeTransform } from '@angular/core'
import { catchError, map, Observable, of } from 'rxjs'
import { Tag } from '../data/tag'
import { SETTINGS_KEYS } from '../data/ui-settings'
import {
PermissionAction,
PermissionsService,
PermissionType,
} from '../services/permissions.service'
import { TagService } from '../services/rest/tag.service'
import { SettingsService } from '../services/settings.service'
import { collapseImpliedAncestors } from '../utils/collapse-tags'
/**
* Drops tag IDs whose descendant is also in the list for when the child's
* Tag badge already shows its parent.
*/
@Pipe({
name: 'collapseTags',
})
export class CollapseTagsPipe implements PipeTransform {
private readonly permissionsService = inject(PermissionsService)
private readonly tagService = inject(TagService)
private readonly settingsService = inject(SettingsService)
transform(ids: number[]): Observable<number[]> {
if (
!ids?.length ||
!this.settingsService.get(SETTINGS_KEYS.TAGS_SHOW_HIERARCHY) ||
!this.permissionsService.currentUserCan(
PermissionAction.View,
PermissionType.Tag
)
) {
return of(ids ?? [])
}
return this.tagService.listAll().pipe(
map((tags) =>
collapseImpliedAncestors(
ids,
new Map<number, Tag>(tags.results.map((t) => [t.id, t]))
)
),
catchError(() => of(ids))
)
}
}
@@ -0,0 +1,47 @@
import { Tag } from '../data/tag'
import { collapseImpliedAncestors, getTagAncestors } from './collapse-tags'
const tags: Tag[] = [
{ id: 1, name: 'Finance' },
{ id: 2, name: 'Account', parent: 1 },
{ id: 3, name: 'Mortgage', parent: 2 },
{ id: 4, name: 'Taxes', parent: 1 },
{ id: 5, name: 'Inbox' },
]
const tagsById = new Map(tags.map((t) => [t.id, t]))
describe('collapseImpliedAncestors', () => {
it('drops ancestors that are implied by a descendant in the list', () => {
expect(collapseImpliedAncestors([1, 2, 3, 4, 5], tagsById)).toEqual([
3, 4, 5,
])
})
it('keeps a parent with none of its children present', () => {
expect(collapseImpliedAncestors([1, 5], tagsById)).toEqual([1, 5])
})
it('keeps unknown (private) ids and handles empty input', () => {
expect(collapseImpliedAncestors([99, 2], tagsById)).toEqual([99, 2])
expect(collapseImpliedAncestors([], tagsById)).toEqual([])
expect(collapseImpliedAncestors(null, tagsById)).toEqual([])
})
})
describe('getTagAncestors', () => {
it('returns ancestors root first', () => {
expect(getTagAncestors(3, tagsById).map((t) => t.id)).toEqual([1, 2])
expect(getTagAncestors(5, tagsById)).toEqual([])
})
it('stops at a parent that is not visible or forms a cycle', () => {
const hidden = new Map<number, Tag>([[6, { id: 6, name: 'x', parent: 99 }]])
expect(getTagAncestors(6, hidden)).toEqual([])
const cyclic = new Map<number, Tag>([
[7, { id: 7, name: 'a', parent: 8 }],
[8, { id: 8, name: 'b', parent: 7 }],
])
expect(getTagAncestors(7, cyclic).map((t) => t.id)).toEqual([7, 8])
})
})
+43
View File
@@ -0,0 +1,43 @@
import { Tag } from '../data/tag'
/**
* Collapses implied ancestor tags from a list of tag IDs such that if a tag's
* ancestor is also present in the list, the ancestor is removed. For example
* [Finance, Account, Mortgage] -> [Mortgage]
*/
export function collapseImpliedAncestors(
ids: number[],
tagsById: Map<number, Tag>
): number[] {
if (!ids?.length) return ids ?? []
const present = new Set(ids)
const implied = new Set<number>()
for (const id of ids) {
const seen = new Set<number>()
let parentId = tagsById.get(id)?.parent
while (parentId && !seen.has(parentId)) {
seen.add(parentId)
if (present.has(parentId)) implied.add(parentId)
parentId = tagsById.get(parentId)?.parent
}
}
return ids.filter((id) => !implied.has(id))
}
/**
* Returns an array of ancestor tags for the given tag ID, starting from the
* root and ending with the immediate parent.
*/
export function getTagAncestors(id: number, tagsById: Map<number, Tag>): Tag[] {
const chain: Tag[] = []
const seen = new Set<number>()
let parentId = tagsById.get(id)?.parent
while (parentId && !seen.has(parentId)) {
seen.add(parentId)
const parent = tagsById.get(parentId)
if (!parent) break
chain.unshift(parent)
parentId = parent.parent
}
return chain
}
+1 -1
View File
@@ -8,7 +8,7 @@ export const environment = {
apiVersion: '10', // match src/paperless/settings.py
appTitle: DEFAULT_APP_TITLE,
tag: 'prod',
version: '3.4.2',
version: '3.5.1',
webSocketHost: window.location.host,
webSocketProtocol: window.location.protocol == 'https:' ? 'wss:' : 'ws:',
webSocketBaseUrl: base_url.pathname + 'ws/',
File diff suppressed because it is too large. Load diff
File diff suppressed because it is too large. Load diff
File diff suppressed because it is too large. Load diff
File diff suppressed because it is too large. Load diff
File diff suppressed because it is too large. Load diff
File diff suppressed because it is too large. Load diff
File diff suppressed because it is too large. Load diff
File diff suppressed because it is too large. Load diff
File diff suppressed because it is too large. Load diff
File diff suppressed because it is too large. Load diff
File diff suppressed because it is too large. Load diff
File diff suppressed because it is too large. Load diff
File diff suppressed because it is too large. Load diff
File diff suppressed because it is too large. Load diff
File diff suppressed because it is too large. Load diff
File diff suppressed because it is too large. Load diff
File diff suppressed because it is too large. Load diff
File diff suppressed because it is too large. Load diff
File diff suppressed because it is too large. Load diff
File diff suppressed because it is too large. Load diff
File diff suppressed because it is too large. Load diff
File diff suppressed because it is too large. Load diff
File diff suppressed because it is too large. Load diff
File diff suppressed because it is too large. Load diff
File diff suppressed because it is too large. Load diff
File diff suppressed because it is too large. Load diff
File diff suppressed because it is too large. Load diff
File diff suppressed because it is too large. Load diff
File diff suppressed because it is too large. Load diff
File diff suppressed because it is too large. Load diff
File diff suppressed because it is too large. Load diff
File diff suppressed because it is too large. Load diff
File diff suppressed because it is too large. Load diff
File diff suppressed because it is too large. Load diff
File diff suppressed because it is too large. Load diff
File diff suppressed because it is too large. Load diff
File diff suppressed because it is too large. Load diff
File diff suppressed because it is too large. Load diff
File diff suppressed because it is too large. Load diff
File diff suppressed because it is too large. Load diff
File diff suppressed because it is too large. Load diff
File diff suppressed because it is too large. Load diff
File diff suppressed because it is too large. Load diff
File diff suppressed because it is too large. Load diff
File diff suppressed because it is too large. Load diff
File diff suppressed because it is too large. Load diff
File diff suppressed because it is too large. Load diff
+69 -126
View File
@@ -39,8 +39,6 @@ from documents.models import Tag
from documents.permissions import PermissionsSpec
from documents.permissions import set_permissions_for_objects
from documents.plugins.helpers import DocumentsStatusManager
from documents.task_tracking import queue_consumption
from documents.task_tracking import queue_processing_tasks
from documents.tasks import bulk_update_documents
from documents.tasks import consume_file
from documents.tasks import remove_document_from_index
@@ -434,12 +432,7 @@ def delete(doc_ids: list[int]) -> Literal["OK"]:
return "OK"
def reprocess(
doc_ids: list[int],
*,
remote_ocr: bool = False,
parent_task_id: str | None = None,
) -> Literal["OK"]:
def reprocess(doc_ids: list[int], *, remote_ocr: bool = False) -> Literal["OK"]:
"""
Re-run parsing for the given documents.
@@ -464,16 +457,6 @@ def reprocess(
.values_list("source_id", flat=True)
.distinct()
)
if parent_task_id is not None:
from documents.task_tracking import queue_reprocessing_tasks
queue_reprocessing_tasks(
parent_task_id,
list(source_ids),
remote_ocr=remote_ocr,
)
return "OK"
for source_id in source_ids:
update_document_content_maybe_archive_file.apply_async(
kwargs={"document_id": source_id, "remote_ocr": remote_ocr},
@@ -538,7 +521,6 @@ def rotate(
source_mode: SourceMode = SourceModeChoices.LATEST_VERSION,
user: User | None = None,
trigger_source: PaperlessTask.TriggerSource = PaperlessTask.TriggerSource.WEB_UI,
parent_task_id: str | None = None,
) -> Literal["OK"]:
logger.info(
f"Attempting to rotate {len(doc_ids)} documents by {degrees} degrees.",
@@ -557,6 +539,7 @@ def rotate(
pair = _resolve_root_and_source_doc(doc, source_mode=source_mode)
docs_by_root_id.setdefault(pair.root_doc.id, pair)
failed_ids = []
for pair in docs_by_root_id.values():
if pair.source_doc.mime_type != "application/pdf":
logger.warning(
@@ -576,22 +559,28 @@ def rotate(
if user is not None:
overrides.actor_id = user.id
queue_consumption(
input_doc=ConsumableDocument(
source=DocumentSource.ConsumeFolder,
original_file=filepath,
root_document_id=pair.root_doc.id,
),
overrides=overrides,
trigger_source=trigger_source,
parent_task_id=parent_task_id,
consume_file.apply_async(
kwargs={
"input_doc": ConsumableDocument(
source=DocumentSource.ConsumeFolder,
original_file=filepath,
root_document_id=pair.root_doc.id,
),
"overrides": overrides,
},
headers={"trigger_source": trigger_source},
)
logger.info(
f"Queued new rotated version for document {pair.root_doc.id} by {degrees} degrees",
)
except Exception as e:
failed_ids.append(pair.root_doc.id)
logger.exception(f"Error rotating document {pair.root_doc.id}: {e}")
raise
if failed_ids:
raise RuntimeError(
f"Rotation failed for document(s): {failed_ids}. Check logs for details.",
)
return "OK"
@@ -605,15 +594,13 @@ def merge(
source_mode: SourceMode = SourceModeChoices.LATEST_VERSION,
user: User | None = None,
trigger_source: PaperlessTask.TriggerSource = PaperlessTask.TriggerSource.WEB_UI,
parent_task_id: str | None = None,
) -> Literal["OK"] | dict:
) -> Literal["OK"]:
logger.info(
f"Attempting to merge {len(doc_ids)} documents into a single document.",
)
qs = Document.objects.select_related("root_document").filter(id__in=doc_ids)
docs_by_id = {doc.id: doc for doc in qs}
affected_docs: list[int] = []
failed_ids: list[int] = []
handoff_asn: int | None = None
with pdf_ops.PdfMerger() as merger:
# use doc_ids to preserve order
@@ -639,14 +626,11 @@ def merge(
if handoff_asn is None and doc.archive_serial_number is not None:
handoff_asn = doc.archive_serial_number
except Exception as e:
failed_ids.append(doc.id)
logger.exception(
f"Error merging document {doc.id}, it will not be included in the merge: {e}",
)
if len(affected_docs) == 0:
logger.warning("No documents were merged")
if parent_task_id is not None:
raise ValueError("None of the selected documents could be merged.")
return "OK"
filepath = (
@@ -689,23 +673,6 @@ def merge(
overrides=overrides,
).set(headers={"trigger_source": trigger_source})
if parent_task_id is not None:
backup = (
release_archive_serial_numbers(affected_docs) if delete_originals else {}
)
queue_processing_tasks(
parent_task_id,
[consume_task],
delete_ids=affected_docs if delete_originals else None,
asn_backup=backup,
)
if failed_ids:
return {
"failed_document_ids": failed_ids,
"error_message": f"Documents {failed_ids} were excluded from the merge.",
}
return "OK"
if delete_originals:
backup = release_archive_serial_numbers(affected_docs)
logger.info(
@@ -725,6 +692,22 @@ def merge(
return "OK"
def validate_merge_as_versions_state(
documents: list[Document],
root_document_id: int,
) -> None:
if any(document.root_document_id is not None for document in documents):
raise ValueError("Only top-level documents can be merged as versions.")
source_ids = [
document.pk for document in documents if document.pk != root_document_id
]
if Document.global_objects.filter(root_document_id__in=source_ids).exists():
raise ValueError(
"Documents with existing versions cannot be merged into another document.",
)
def merge_as_versions(
doc_ids: list[int],
*,
@@ -742,15 +725,8 @@ def merge_as_versions(
)
documents_by_id = {document.id: document for document in documents}
if any(document.root_document_id is not None for document in documents):
raise ValueError("Only top-level documents can be merged as versions.")
validate_merge_as_versions_state(documents, root_document_id)
source_ids = [doc_id for doc_id in doc_ids if doc_id != root_document_id]
if Document.global_objects.filter(root_document_id__in=source_ids).exists():
raise ValueError(
"Documents with existing versions cannot be merged into another document.",
)
root_document = documents_by_id[root_document_id]
next_version_index = (
Document.global_objects.filter(
@@ -850,7 +826,6 @@ def split(
source_mode: SourceMode = SourceModeChoices.LATEST_VERSION,
user: User | None = None,
trigger_source: PaperlessTask.TriggerSource = PaperlessTask.TriggerSource.WEB_UI,
parent_task_id: str | None = None,
) -> Literal["OK"]:
logger.info(
f"Attempting to split document {doc_ids[0]} into {len(pages)} documents",
@@ -894,17 +869,7 @@ def split(
).set(headers={"trigger_source": trigger_source}),
)
if parent_task_id is not None:
backup = (
release_archive_serial_numbers([doc.id]) if delete_originals else {}
)
queue_processing_tasks(
parent_task_id,
consume_tasks,
delete_ids=[doc.id] if delete_originals else None,
asn_backup=backup,
)
elif delete_originals:
if delete_originals:
backup = release_archive_serial_numbers([doc.id])
logger.info(
"Queueing removal of original document after consumption of the split documents",
@@ -936,7 +901,6 @@ def delete_pages(
source_mode: SourceMode = SourceModeChoices.LATEST_VERSION,
user: User | None = None,
trigger_source: PaperlessTask.TriggerSource = PaperlessTask.TriggerSource.WEB_UI,
parent_task_id: str | None = None,
) -> Literal["OK"]:
logger.info(
f"Attempting to delete pages {pages} from {len(doc_ids)} documents",
@@ -955,15 +919,16 @@ def delete_pages(
overrides = DocumentMetadataOverrides().from_document(pair.root_doc)
if user is not None:
overrides.actor_id = user.id
queue_consumption(
input_doc=ConsumableDocument(
source=DocumentSource.ConsumeFolder,
original_file=filepath,
root_document_id=pair.root_doc.id,
),
overrides=overrides,
trigger_source=trigger_source,
parent_task_id=parent_task_id,
consume_file.apply_async(
kwargs={
"input_doc": ConsumableDocument(
source=DocumentSource.ConsumeFolder,
original_file=filepath,
root_document_id=pair.root_doc.id,
),
"overrides": overrides,
},
headers={"trigger_source": trigger_source},
)
logger.info(
f"Queued new version for document {pair.root_doc.id} after deleting pages {pages}",
@@ -985,7 +950,6 @@ def edit_pdf(
source_mode: SourceMode = SourceModeChoices.LATEST_VERSION,
user: User | None = None,
trigger_source: PaperlessTask.TriggerSource = PaperlessTask.TriggerSource.WEB_UI,
parent_task_id: str | None = None,
) -> Literal["OK"]:
"""
Operations is a list of dictionaries describing the final PDF pages.
@@ -1030,15 +994,16 @@ def edit_pdf(
if user is not None:
overrides.owner_id = user.id
overrides.actor_id = user.id
queue_consumption(
input_doc=ConsumableDocument(
source=DocumentSource.ConsumeFolder,
original_file=filepath,
root_document_id=pair.root_doc.id,
),
overrides=overrides,
trigger_source=trigger_source,
parent_task_id=parent_task_id,
consume_file.apply_async(
kwargs={
"input_doc": ConsumableDocument(
source=DocumentSource.ConsumeFolder,
original_file=filepath,
root_document_id=pair.root_doc.id,
),
"overrides": overrides,
},
headers={"trigger_source": trigger_source},
)
else:
version_filepaths = pdf_ops.build_pdfs(
@@ -1078,17 +1043,7 @@ def edit_pdf(
).set(headers={"trigger_source": trigger_source}),
)
if parent_task_id is not None:
backup = (
release_archive_serial_numbers([doc.id]) if delete_original else {}
)
queue_processing_tasks(
parent_task_id,
consume_tasks,
delete_ids=[doc.id] if delete_original else None,
asn_backup=backup,
)
elif delete_original:
if delete_original:
backup = release_archive_serial_numbers([doc.id])
try:
chord(
@@ -1122,7 +1077,6 @@ def remove_password(
source_mode: SourceMode = SourceModeChoices.LATEST_VERSION,
user: User | None = None,
trigger_source: PaperlessTask.TriggerSource = PaperlessTask.TriggerSource.WEB_UI,
parent_task_id: str | None = None,
source_paths_by_id: Mapping[int, Path] | None = None,
) -> Literal["OK"]:
"""
@@ -1165,15 +1119,16 @@ def remove_password(
if user is not None:
overrides.owner_id = user.id
overrides.actor_id = user.id
queue_consumption(
input_doc=ConsumableDocument(
source=DocumentSource.ConsumeFolder,
original_file=filepath,
root_document_id=pair.root_doc.id,
),
overrides=overrides,
trigger_source=trigger_source,
parent_task_id=parent_task_id,
consume_file.apply_async(
kwargs={
"input_doc": ConsumableDocument(
source=DocumentSource.ConsumeFolder,
original_file=filepath,
root_document_id=pair.root_doc.id,
),
"overrides": overrides,
},
headers={"trigger_source": trigger_source},
)
else:
consume_tasks = []
@@ -1196,19 +1151,7 @@ def remove_password(
).set(headers={"trigger_source": trigger_source}),
)
if parent_task_id is not None:
backup = (
release_archive_serial_numbers([doc.id])
if delete_original
else {}
)
queue_processing_tasks(
parent_task_id,
consume_tasks,
delete_ids=[doc.id] if delete_original else None,
asn_backup=backup,
)
elif delete_original:
if delete_original:
chord(
header=consume_tasks,
body=delete.si([doc.id]),
+8 -15
View File
@@ -1,5 +1,6 @@
"""Queued document bulk operations."""
from contextlib import nullcontext
from typing import Any
from auditlog.context import set_actor
@@ -89,9 +90,8 @@ def snapshot_field(doc_ids: list[int], field: str) -> dict[int, Any]:
return values
@shared_task(bind=True)
@shared_task
def execute_bulk_operation(
self,
operation: str,
document_ids: list[int],
parameters: dict,
@@ -113,25 +113,22 @@ def execute_bulk_operation(
if operation == "run_workflow" and not user.has_perm("documents.view_workflow"):
raise PermissionDenied("Insufficient permissions to run this workflow")
actor_context = set_actor(user) if settings.AUDIT_LOG_ENABLED else nullcontext()
if operation not in MODIFIED_FIELDS:
execution_parameters = parameters.copy()
if (
operation in TRIGGER_SOURCE_OPERATIONS | {"reprocess"}
and self.request.id is not None
):
execution_parameters["parent_task_id"] = self.request.id
if operation in USER_OPERATIONS:
execution_parameters["user"] = user
if operation in TRIGGER_SOURCE_OPERATIONS:
execution_parameters["trigger_source"] = trigger_source
modified_field = LEGACY_MODIFIED_FIELDS.get(operation)
with set_actor(user):
with actor_context:
old_values = (
snapshot_field(document_ids, modified_field)
if settings.AUDIT_LOG_ENABLED and modified_field
else {}
)
result = method(document_ids, **execution_parameters)
method(document_ids, **execution_parameters)
if settings.AUDIT_LOG_ENABLED and modified_field:
from auditlog.models import LogEntry
@@ -146,17 +143,13 @@ def execute_bulk_operation(
actor=user,
additional_data={"reason": f"Bulk edit: {operation}"},
)
return {
"operation": operation,
"document_count": len(document_ids),
**(result if isinstance(result, dict) else {}),
}
return {"operation": operation, "document_count": len(document_ids)}
affected_ids: list[int] = []
modified_field = MODIFIED_FIELDS[operation]
# Keep metadata writes and their audit entries together. Follow-up processing
# runs after commit, within this same tracked task.
with set_actor(user), transaction.atomic():
with actor_context, transaction.atomic():
old_values = (
snapshot_field(document_ids, modified_field)
if settings.AUDIT_LOG_ENABLED and modified_field
+45 -3
View File
@@ -3,7 +3,6 @@ from __future__ import annotations
import logging
import re
from fnmatch import fnmatch
from fnmatch import translate as fnmatch_translate
from typing import TYPE_CHECKING
from rest_framework import serializers
@@ -548,6 +547,48 @@ def existing_document_matches_workflow(
return (True, None)
def _glob_to_portable_regex(pattern: str) -> str:
"""
Translate an fnmatch-style glob into an anchored regex which is accepted by
Python, PostgreSQL (ARE), and MariaDB (PCRE).
fnmatch.translate() can't be used, as its output (`(?s:...)\\Z`) uses syntax
PostgreSQL rejects. Supports `*`, `?`, `[seq]` and `[!seq]`.
"""
out: list[str] = []
i = 0
n = len(pattern)
while i < n:
c = pattern[i]
i += 1
if c == "*":
out.append(".*")
elif c == "?":
out.append(".")
elif c == "[":
j = i
if j < n and pattern[j] == "!":
j += 1
if j < n and pattern[j] == "]":
j += 1
while j < n and pattern[j] != "]":
j += 1
if j >= n:
# No closing bracket, treat as a literal
out.append(r"\[")
else:
body = pattern[i:j]
i = j + 1
negate = body.startswith("!")
if negate:
body = body[1:]
body = "".join(f"\\{ch}" if ch in "\\[^" else ch for ch in body)
out.append(f"[{'^' if negate else ''}{body}]")
else:
out.append(re.escape(c))
return f"^{''.join(out)}$"
def prefilter_documents_by_workflowtrigger(
documents: QuerySet[Document],
trigger: WorkflowTrigger,
@@ -628,8 +669,9 @@ def prefilter_documents_by_workflowtrigger(
documents = documents.annotate(**annotations).filter(custom_field_q)
if trigger.filter_filename:
regex = fnmatch_translate(trigger.filter_filename).lstrip("^").rstrip("$")
documents = documents.filter(original_filename__iregex=regex)
documents = documents.filter(
original_filename__iregex=_glob_to_portable_regex(trigger.filter_filename),
)
return documents
@@ -1,30 +0,0 @@
# Generated by Django 5.2.18 on 2026-11-14 19:59
import django.db.models.deletion
from django.db import migrations
from django.db import models
class Migration(migrations.Migration):
dependencies = [
("documents", "0030_note_format"),
]
operations = [
migrations.AddField(
model_name="paperlesstask",
name="dispatched",
field=models.BooleanField(default=False),
),
migrations.AddField(
model_name="paperlesstask",
name="parent",
field=models.ForeignKey(
blank=True,
null=True,
on_delete=django.db.models.deletion.SET_NULL,
related_name="children",
to="documents.paperlesstask",
),
),
]
-10
View File
@@ -970,16 +970,6 @@ class PaperlessTask(ModelWithOwner):
help_text=_("Structured result data from task execution"),
)
parent = models.ForeignKey(
"self",
null=True,
blank=True,
on_delete=models.SET_NULL,
related_name="children",
)
dispatched = models.BooleanField(default=False)
# Acknowledgment
acknowledged = models.BooleanField(
default=False,
+11 -2
View File
@@ -1717,8 +1717,17 @@ class MergeDocumentsAsVersionsSerializer(DocumentListSerializer):
"root_document_id must be one of the selected documents.",
)
# Version state checks (e.g., whether a document is already part of a version chain)
# moved to merge_as_versions() under row locks, since it could change between queueing and execution
# Rechecked under row locks during execution in case document state changes.
selected_documents = list(
Document.objects.filter(id__in=documents).only("id", "root_document"),
)
try:
bulk_edit.validate_merge_as_versions_state(
selected_documents,
attrs["root_document_id"],
)
except ValueError as error:
raise serializers.ValidationError(str(error)) from error
return attrs
-44
View File
@@ -56,8 +56,6 @@ from documents.models import WorkflowRun
from documents.models import WorkflowTrigger
from documents.permissions import restrict_queryset_to_visible
from documents.plugins.helpers import DocumentsStatusManager
from documents.task_tracking import finish_dispatch
from documents.task_tracking import update_parent
from documents.templating.utils import convert_format_str_to_template_format
from documents.utils import compute_checksum
from documents.utils import copy_file_with_basic_stats
@@ -1079,7 +1077,6 @@ TRACKED_TASKS: dict[str, PaperlessTask.TaskType] = {
"documents.tasks.update_document_content_maybe_archive_file": PaperlessTask.TaskType.REPROCESS_DOCUMENT,
"documents.tasks.build_share_link_bundle": PaperlessTask.TaskType.BUILD_SHARE_LINK,
"documents.bulk_edit.delete": PaperlessTask.TaskType.BULK_DELETE,
"documents.task_tracking.finish_file_operation": PaperlessTask.TaskType.BULK_DELETE,
"documents.tasks.apply_ai_suggestions": PaperlessTask.TaskType.APPLY_AI_SUGGESTIONS,
BULK_OPERATION_TASK_NAME: PaperlessTask.TaskType.BULK_UPDATE,
}
@@ -1128,8 +1125,6 @@ def _extract_input_data(
data["overrides"] = override_dict
return data
if task_type == PaperlessTask.TaskType.REPROCESS_DOCUMENT:
return {"document_id": task_kwargs["document_id"]}
if task_type == PaperlessTask.TaskType.MAIL_FETCH:
account_ids = task_kwargs.get("account_ids")
if account_ids is not None:
@@ -1217,19 +1212,6 @@ def before_task_publish_handler(
input_data = _extract_input_data(task_type, task_kwargs)
owner_id = _extract_owner_id(task_type, task_kwargs)
trigger_source = _determine_trigger_source(headers)
# Consumption can fan out again (e.g. barcode splitting). Keep those
# descendants attached so their caller cannot finish before them.
parent = None
if task_type == PaperlessTask.TaskType.CONSUME_FILE and headers.get(
"parent_id",
):
parent = PaperlessTask.objects.filter(
task_id=headers["parent_id"],
parent__isnull=False,
).first()
if parent is not None:
owner_id = parent.owner_id
trigger_source = parent.trigger_source
# A retried task is republished with the same task_id, so this fires
# again for it; get_or_create keeps the original PENDING record
@@ -1242,7 +1224,6 @@ def before_task_publish_handler(
"status": PaperlessTask.Status.PENDING,
"input_data": input_data,
"owner_id": owner_id,
"parent": parent,
},
)
except Exception: # pragma: no cover
@@ -1328,10 +1309,6 @@ def task_postrun_handler(
except PaperlessTask.DoesNotExist:
return
if task_instance.children.exists():
finish_dispatch(task_instance, retval if isinstance(retval, dict) else {})
return
task_instance.status = new_status
task_instance.date_done = now
changed_fields = ["status", "date_done"]
@@ -1354,21 +1331,7 @@ def task_postrun_handler(
task_instance.status = PaperlessTask.Status.FAILURE
changed_fields.append("status")
if (
task_instance.parent_id is not None
and task_instance.task_type == PaperlessTask.TaskType.CONSUME_FILE
and (not isinstance(retval, dict) or "document_id" not in retval)
and not (isinstance(retval, dict) and "duplicate_of" in retval)
):
task_instance.status = PaperlessTask.Status.FAILURE
task_instance.result_data = {
**(retval if isinstance(retval, dict) else {}),
"error_message": "Consumption stopped without creating a document.",
}
changed_fields.extend(["status", "result_data"])
task_instance.save(update_fields=changed_fields)
update_parent(task_instance)
if task and task.name == BULK_OPERATION_TASK_NAME:
_send_bulk_operation_status(task_instance)
except Exception: # pragma: no cover
@@ -1425,10 +1388,6 @@ def task_failure_handler(
}
task_qs = PaperlessTask.objects.filter(task_id=task_id)
parent = task_qs.first()
if parent is not None and parent.children.exists():
finish_dispatch(parent, result_data)
return
task_instance = task_qs.values("date_started", "date_created").first()
if task_instance:
date_started = task_instance["date_started"]
@@ -1440,7 +1399,6 @@ def task_failure_handler(
date_started - date_created
).total_seconds()
task_qs.update(**update_fields)
update_parent(task_qs.get())
if sender and sender.name == BULK_OPERATION_TASK_NAME:
_send_bulk_operation_status(task_qs.first())
except Exception: # pragma: no cover
@@ -1487,8 +1445,6 @@ def task_revoked_handler(
"reason": (reason + " Check worker logs for details."),
},
)
if task_instance := task_qs.first():
update_parent(task_instance)
if sender and sender.name == BULK_OPERATION_TASK_NAME:
_send_bulk_operation_status(task_qs.first())
except Exception: # pragma: no cover
-242
View File
@@ -1,242 +0,0 @@
"""Completion tracking for operations which dispatch document processing tasks."""
from __future__ import annotations
import uuid
from typing import TYPE_CHECKING
from celery import shared_task
from django.db import transaction
from django.utils import timezone
from documents.models import PaperlessTask
if TYPE_CHECKING:
from celery.canvas import Signature
from documents.data_models import ConsumableDocument
from documents.data_models import DocumentMetadataOverrides
def queue_consumption(
*,
input_doc: ConsumableDocument,
overrides: DocumentMetadataOverrides,
trigger_source: str,
parent_task_id: str | None,
) -> None:
from documents.tasks import consume_file
kwargs = {"input_doc": input_doc, "overrides": overrides}
if parent_task_id is None:
consume_file.apply_async(
kwargs=kwargs,
headers={"trigger_source": trigger_source},
)
else:
queue_processing_tasks(parent_task_id, [consume_file.s(**kwargs)])
def queue_reprocessing_tasks(
parent_task_id: str,
document_ids: list[int],
*,
remote_ocr: bool,
) -> None:
from documents.tasks import update_document_content_maybe_archive_file
queue_processing_tasks(
parent_task_id,
[
update_document_content_maybe_archive_file.s(
document_id=document_id,
remote_ocr=remote_ocr,
)
for document_id in document_ids
],
)
def queue_processing_tasks(
parent_task_id: str,
signatures: list[Signature],
*,
delete_ids: list[int] | None = None,
asn_backup: dict | None = None,
) -> None:
from documents.signals.handlers import TRACKED_TASKS
from documents.signals.handlers import _extract_input_data
parent = PaperlessTask.objects.get(task_id=parent_task_id)
children = [
PaperlessTask(
task_id=str(uuid.uuid4()),
task_type=TRACKED_TASKS[signature.task],
trigger_source=parent.trigger_source,
owner_id=parent.owner_id,
parent=parent,
input_data=_extract_input_data(
TRACKED_TASKS[signature.task],
signature.kwargs,
),
)
for signature in signatures
]
# Dependencies and their final step must be registered together.
with transaction.atomic():
PaperlessTask.objects.bulk_create(children)
if delete_ids:
PaperlessTask.objects.create(
task_id=str(uuid.uuid4()),
task_type=PaperlessTask.TaskType.BULK_DELETE,
owner_id=parent.owner_id,
trigger_source=parent.trigger_source,
parent=parent,
input_data={
"dependencies": [child.pk for child in children],
"delete_ids": delete_ids,
"asn_backup": asn_backup or {},
},
)
for index, (signature, child) in enumerate(zip(signatures, children, strict=True)):
try:
signature.apply_async(
task_id=child.task_id,
headers={"trigger_source": parent.trigger_source},
)
except Exception:
PaperlessTask.objects.filter(
pk__in=[task.pk for task in children[index:]],
).update(
status=PaperlessTask.Status.FAILURE,
date_done=timezone.now(),
result_data={"error_message": "Could not publish processing task."},
)
raise
@shared_task
def finish_file_operation(
dependencies: list[int],
delete_ids: list[int],
asn_backup: dict,
) -> dict[str, str]:
from documents.bulk_edit import delete
from documents.bulk_edit import restore_archive_serial_numbers
statuses = list(
PaperlessTask.objects.filter(pk__in=dependencies).values_list(
"status",
flat=True,
),
)
if len(statuses) != len(dependencies):
raise ValueError(
"Processing task records are missing; originals were preserved.",
)
if any(status != PaperlessTask.Status.SUCCESS for status in statuses):
restore_archive_serial_numbers(asn_backup)
return {"reason": "Originals preserved; archive serial numbers restored."}
delete(delete_ids)
return {"reason": "Originals moved to trash."}
def _publish_finalizer(task: PaperlessTask) -> None:
try:
finish_file_operation.apply_async(
task_id=task.task_id,
kwargs=task.input_data,
headers={"trigger_source": task.trigger_source},
)
except Exception as error:
from documents.signals.handlers import task_failure_handler
task_failure_handler(
sender=finish_file_operation,
task_id=task.task_id,
exception=error,
)
def _schedule_finalizers(parent: PaperlessTask) -> None:
for task in parent.children.filter(
task_type=PaperlessTask.TaskType.BULK_DELETE,
status=PaperlessTask.Status.PENDING,
):
dependencies = task.input_data.get("dependencies")
if dependencies is None or task.result_data:
continue
if (
PaperlessTask.objects.filter(pk__in=dependencies)
.exclude(
status__in=PaperlessTask.COMPLETE_STATUSES,
)
.exists()
):
continue
task.result_data = {"scheduled": True}
task.save(update_fields=["result_data"])
transaction.on_commit(lambda task=task: _publish_finalizer(task))
def finish_dispatch(task: PaperlessTask, result_data: dict) -> None:
"""Record the scheduling outcome without completing an active parent."""
with transaction.atomic():
parent = PaperlessTask.objects.select_for_update().get(pk=task.pk)
if parent.is_complete:
return
parent.dispatched = True
parent.result_data = result_data
parent.save(update_fields=["dispatched", "result_data"])
_finalize_parent(parent)
def update_parent(task: PaperlessTask) -> None:
if task.parent_id is None:
return
with transaction.atomic():
parent = PaperlessTask.objects.select_for_update().get(pk=task.parent_id)
_finalize_parent(parent)
def _finalize_parent(parent: PaperlessTask) -> None:
if parent.is_complete or not parent.dispatched:
return
_schedule_finalizers(parent)
statuses = list(parent.children.values_list("status", flat=True))
if any(status not in PaperlessTask.COMPLETE_STATUSES for status in statuses):
return
succeeded = statuses.count(PaperlessTask.Status.SUCCESS)
failed = len(statuses) - succeeded
result_data = parent.result_data or {}
if failed and not result_data.get("error_message"):
result_data["error_message"] = (
f"{succeeded} child task(s) succeeded; {failed} failed or were revoked. "
"See child tasks for details."
)
parent.status = (
PaperlessTask.Status.FAILURE
if result_data.get("error_message")
else PaperlessTask.Status.SUCCESS
)
parent.result_data = {
**parent.input_data,
**result_data,
"succeeded_count": succeeded,
"failed_count": failed,
}
parent.date_done = timezone.now()
if parent.date_started:
parent.duration_seconds = (
parent.date_done - parent.date_started
).total_seconds()
parent.wait_time_seconds = (
parent.date_started - parent.date_created
).total_seconds()
parent.save()
from documents.signals.handlers import _send_bulk_operation_status
if "operation" in parent.input_data:
transaction.on_commit(lambda: _send_bulk_operation_status(parent))
update_parent(parent)
+3 -3
View File
@@ -394,10 +394,11 @@ def update_document_content_maybe_archive_file(
)
if not parser_class:
raise ValueError(
logger.error(
f"No parser found for mime type {mime_type}, cannot "
f"reprocess document {document} (ID: {document_id})",
f"archive document {document} (ID: {document_id})",
)
return
with parser_class() as parser:
parser.configure(ParserContext())
@@ -511,7 +512,6 @@ def update_document_content_maybe_archive_file(
logger.exception(
f"Error while parsing document {document} (ID: {document_id})",
)
raise
@shared_task
+81 -2
View File
@@ -4,14 +4,18 @@ from typing import TYPE_CHECKING
from unittest import mock
import pytest
from asgiref.sync import async_to_sync
from rest_framework import status
from rest_framework.test import APIClient
from rest_framework.test import APITestCase
from documents.views import _iterate_in_sync_thread
from paperless_testing.factories import UserFactory
from paperless_testing.permissions import grant_global
if TYPE_CHECKING:
from collections.abc import Generator
from django.contrib.auth.models import User
from pytest_mock import MockerFixture
@@ -62,7 +66,7 @@ class TestChatStreamingViewInputValidation(APITestCase):
),
mock.patch(
"paperless_ai.chat.stream_chat_with_documents",
return_value=iter(chunks),
return_value=(c for c in chunks),
),
):
resp = self.client.post(
@@ -74,7 +78,11 @@ class TestChatStreamingViewInputValidation(APITestCase):
assert resp.status_code == status.HTTP_200_OK
assert not resp.has_header("Content-Encoding")
assert list(resp.streaming_content) == [c.encode() for c in chunks]
async def read_parts() -> list[bytes]:
return [part async for part in resp.streaming_content]
assert async_to_sync(read_parts)() == [c.encode() for c in chunks]
def test_missing_question_is_rejected(self) -> None:
with mock.patch(
@@ -155,3 +163,74 @@ class TestChatStreamingViewUnrestrictedFlag:
assert (
mocked_stream_chat.call_args.kwargs["unrestricted"] is expected_unrestricted
)
class TestIterateInSyncThread:
@pytest.fixture
def events(self) -> list[str]:
return []
@pytest.fixture
def chunks(self, events: list[str]) -> Generator[str, None, None]:
def generate() -> Generator[str, None, None]:
try:
for chunk in ("first", "second"):
events.append(f"produced {chunk}")
yield chunk
finally:
events.append("closed")
return generate()
def test_yields_each_chunk_before_pulling_the_next(
self,
chunks: Generator[str, None, None],
events: list[str],
) -> None:
"""
GIVEN:
- A sync generator wrapped for async iteration
WHEN:
- The wrapper is consumed
THEN:
- Each chunk is delivered before the next one is generated,
rather than the whole generator being drained up front
"""
async def consume() -> None:
async for chunk in _iterate_in_sync_thread(chunks):
events.append(f"received {chunk}")
async_to_sync(consume)()
assert events == [
"produced first",
"received first",
"produced second",
"received second",
"closed",
]
def test_closing_early_closes_the_source_generator(
self,
chunks: Generator[str, None, None],
events: list[str],
) -> None:
"""
GIVEN:
- A sync generator wrapped for async iteration
WHEN:
- The consumer closes the wrapper after the first chunk, as
happens when the client disconnects
THEN:
- The source generator is closed, so its cleanup runs
"""
async def consume_one() -> None:
wrapper = _iterate_in_sync_thread(chunks)
await anext(wrapper)
await wrapper.aclose()
async_to_sync(consume_one)()
assert events == ["produced first", "closed"]
Loaded 100 of 164 files, more files were not shown because too many files have changed in this diff. Show more