Compare commits

..
Author SHA1 Message Date
shamoon 3508877a33 Update api.md 2026-11-14 12:31:08 -08:00
shamoon fea012d5c0 Apply to the rest of the operations 2026-11-14 12:30:48 -08:00
shamoon e2aa45da4e Fix a couple dispatched renames 2026-11-14 12:21:25 -08:00
shamoon 5ad0773446 Buld this parent task into the model, dispatched. Hook up for reprocess 2026-11-14 12:16:02 -08:00
shamoon 13202f52ad Raise reprocess errors 2026-11-14 11:54:40 -08:00
shamoon 9a9d4284d7 Couple more things for sonar 2026-11-14 11:51:42 -08:00
shamoon 6043dd1c81 Be more careful about how we handle errors wrt open docs 2026-11-14 11:51:42 -08:00
shamoon 058e27e3ef Reprot more accurate notification for tasks with follow-ups 2026-11-14 11:51:42 -08:00
shamoon 6f5a192967 Propagate these errors now so they make it to the task 2026-11-14 11:51:41 -08:00
shamoon bb775a16c1 Move the merge as version heirarchy checks to the bulk edit method 2026-11-14 11:51:41 -08:00
shamoon 6d26d3f5ca Sonar loves a readonly class member 2026-11-14 10:53:49 -08:00
shamoon 2131feac73 Random test cleanup stuff 2026-11-14 09:36:10 -08:00
shamoon e2705768b4 Make the popups more helpful 2026-11-14 09:34:01 -08:00
shamoon ce3c4e0855 Yay make the task list useful 2026-11-14 09:34:01 -08:00
shamoon 3a241e51d9 Better info for bulk tasks 2026-11-14 09:34:01 -08:00
shamoon 020a2c8fc0 Dont show view task on success popups 2026-11-14 09:34:00 -08:00
shamoon b533bfe4b5 We need a tasks notification service 2026-11-14 09:34:00 -08:00
shamoon 813dfe487f We have to make this generic-er 2026-11-14 09:34:00 -08:00
shamoon 557afa6454 Move doc detail actions over 2026-11-14 09:33:59 -08:00
shamoon c72473f7bc Fix delete and undo 2026-11-14 09:33:59 -08:00
shamoon 7d79b544ae use the queued operation pathway across teh frontend 2026-11-14 09:33:58 -08:00
shamoon a9fef2b931 Update the api docs 2026-11-14 09:33:58 -08:00
shamoon adbd7561d4 Tests for the actions 2026-11-14 09:33:58 -08:00
shamoon 9f694cdd2d queue the dedicated document actions 2026-11-14 09:33:57 -08:00
shamoon 7d78716069 and add some 2026-11-14 09:33:57 -08:00
shamoon 9523d158a8 Fix the old tests 2026-11-14 09:33:57 -08:00
shamoon 91316df9b7 Move the other bulk edit actions through the new taks tracking route 2026-11-14 09:33:56 -08:00
shamoon 7e67013c42 Ensure from_webui 2026-11-14 09:33:56 -08:00
shamoon 2a0c1249fa Raise error on trash deletion so it can be tracked 2026-11-14 09:33:56 -08:00
164 changed files with 14302 additions and 27813 deletions

No files matched your search

+7 -7
View File
@@ -285,14 +285,14 @@ The API supports various bulk-editing operations which are executed asynchronous
### Documents
Document bulk operations, including the dedicated document action endpoints, return
`{"result":"OK","task_id":"…"}` when queued. Use `/api/tasks/?task_id={uuid}`
to check the task outcome. Actions that create or reprocess files may start further
consumption or reprocessing tasks and completion of the bulk task means those tasks
were scheduled, not that their file processing has finished.
Document bulk operations return `{"result":"OK","task_id":"…"}` when queued.
Check `/api/tasks/?task_id={uuid}` for the outcome, including any follow-up processing,
deletion, or recovery. Failed operations may leave partial changes applied.
For bulk operations on documents, use the endpoint `/api/documents/bulk_edit/` which accepts
a json payload of the format:
For operations with child tasks, `result_data` includes `succeeded_count` and
`failed_count` (task counts). Merges report skipped inputs in `failed_document_ids`.
The `/api/documents/bulk_edit/` endpoint accepts:
```json
{
-116
View File
@@ -1,121 +1,5 @@
# Changelog
## paperless-ngx 3.5.1
### Changes
- Tweak: adjust chat timing for streaming [@shamoon](https://github.com/shamoon) ([#14805](https://github.com/paperless-ngx/paperless-ngx/pull/14805))
### Bug Fixes
- Fix: dont use set\_actor if audit log disabled in tasks [@shamoon](https://github.com/shamoon) ([#14808](https://github.com/paperless-ngx/paperless-ngx/pull/14808))
- Fix: stream AI chat answers under ASGI [@stumpylog](https://github.com/stumpylog) ([#14803](https://github.com/paperless-ngx/paperless-ngx/pull/14803))
- Tweak/fix: retain bulk edit selection after queuing operation [@shamoon](https://github.com/shamoon) ([#14797](https://github.com/paperless-ngx/paperless-ngx/pull/14797))
### All App Changes
<details>
<summary>4 changes</summary>
- Fix: dont use set\_actor if audit log disabled in tasks [@shamoon](https://github.com/shamoon) ([#14808](https://github.com/paperless-ngx/paperless-ngx/pull/14808))
- Tweak: adjust chat timing for streaming [@shamoon](https://github.com/shamoon) ([#14805](https://github.com/paperless-ngx/paperless-ngx/pull/14805))
- Fix: stream AI chat answers under ASGI [@stumpylog](https://github.com/stumpylog) ([#14803](https://github.com/paperless-ngx/paperless-ngx/pull/14803))
- Tweak/fix: retain bulk edit selection after queuing operation [@shamoon](https://github.com/shamoon) ([#14797](https://github.com/paperless-ngx/paperless-ngx/pull/14797))
</details>
## paperless-ngx 3.5.0
### Features / Enhancements
- Chorehancement: move all bulk operations to async hand-off [@shamoon](https://github.com/shamoon) ([#14773](https://github.com/paperless-ngx/paperless-ngx/pull/14773))
- Enhancement: render document notes as markdown [@zwif](https://github.com/zwif) ([#14626](https://github.com/paperless-ngx/paperless-ngx/pull/14626))
- Feature: manual workflow trigger [@shamoon](https://github.com/shamoon) ([#14704](https://github.com/paperless-ngx/paperless-ngx/pull/14704))
- Enhancement: make bulk metadata operations run async [@shamoon](https://github.com/shamoon) ([#14711](https://github.com/paperless-ngx/paperless-ngx/pull/14711))
- Fixhancement: add automatic clearing of sessions [@shamoon](https://github.com/shamoon) ([#14722](https://github.com/paperless-ngx/paperless-ngx/pull/14722))
- Tweak: chat reference tweaks [@shamoon](https://github.com/shamoon) ([#14723](https://github.com/paperless-ngx/paperless-ngx/pull/14723))
- Fixhancement: rework saved view widget queue processing to expected [@shamoon](https://github.com/shamoon) ([#14699](https://github.com/paperless-ngx/paperless-ngx/pull/14699))
### Bug Fixes
- Fix: use a constant password mask and consolidate the no-change check [@stumpylog](https://github.com/stumpylog) ([#14784](https://github.com/paperless-ngx/paperless-ngx/pull/14784))
- Fix: use a PostgreSQL-compatible regex for scheduled workflow filename filters [@stumpylog](https://github.com/stumpylog) ([#14783](https://github.com/paperless-ngx/paperless-ngx/pull/14783))
- Fix: get users for saved view create dialog [@shamoon](https://github.com/shamoon) ([#14769](https://github.com/paperless-ngx/paperless-ngx/pull/14769))
- Fix: Allow AI chat filters over more than 32700 documents [@stumpylog](https://github.com/stumpylog) ([#14754](https://github.com/paperless-ngx/paperless-ngx/pull/14754))
- Fix: Enforce CRLF when writing an email message to the file [@stumpylog](https://github.com/stumpylog) ([#14745](https://github.com/paperless-ngx/paperless-ngx/pull/14745))
- Fix: restore image without archive version thumbnail generation [@shamoon](https://github.com/shamoon) ([#14741](https://github.com/paperless-ngx/paperless-ngx/pull/14741))
- Fix: Blacklist certain OCR user arguments that paperless itself uses [@stumpylog](https://github.com/stumpylog) ([#14734](https://github.com/paperless-ngx/paperless-ngx/pull/14734))
- Fix: remove redundant ui task text filters [@shamoon](https://github.com/shamoon) ([#14736](https://github.com/paperless-ngx/paperless-ngx/pull/14736))
- Fix: fix allowed\_hosts default [@shamoon](https://github.com/shamoon) ([#14735](https://github.com/paperless-ngx/paperless-ngx/pull/14735))
- Fixhancement: add automatic clearing of sessions [@shamoon](https://github.com/shamoon) ([#14722](https://github.com/paperless-ngx/paperless-ngx/pull/14722))
- Fix: two dropdown-related UI fixes [@shamoon](https://github.com/shamoon) ([#14717](https://github.com/paperless-ngx/paperless-ngx/pull/14717))
- Fix: use filtered documents for statistics ASN max [@shamoon](https://github.com/shamoon) ([#14710](https://github.com/paperless-ngx/paperless-ngx/pull/14710))
- Fix: make config view permissions consistent [@shamoon](https://github.com/shamoon) ([#14709](https://github.com/paperless-ngx/paperless-ngx/pull/14709))
- Fix: release the LLM index read lock before streaming the chat answer [@stumpylog](https://github.com/stumpylog) ([#14674](https://github.com/paperless-ngx/paperless-ngx/pull/14674))
- Fix: retrieve once per document chat question [@stumpylog](https://github.com/stumpylog) ([#14673](https://github.com/paperless-ngx/paperless-ngx/pull/14673))
- Fixhancement: rework saved view widget queue processing to expected [@shamoon](https://github.com/shamoon) ([#14699](https://github.com/paperless-ngx/paperless-ngx/pull/14699))
- Fix: apply PDF/A conversion once for mail archives [@shamoon](https://github.com/shamoon) ([#14698](https://github.com/paperless-ngx/paperless-ngx/pull/14698))
- Fix default MFA trusted AMR setting [@shamoon](https://github.com/shamoon) ([#14693](https://github.com/paperless-ngx/paperless-ngx/pull/14693))
- Fix: check descendant tag permissions on delete [@shamoon](https://github.com/shamoon) ([#14687](https://github.com/paperless-ngx/paperless-ngx/pull/14687))
- Fix: lock rows for first-user signup [@shamoon](https://github.com/shamoon) ([#14688](https://github.com/paperless-ngx/paperless-ngx/pull/14688))
- Fix: prevent ID badge wrapping under title on mobile [@shamoon](https://github.com/shamoon) ([#14682](https://github.com/paperless-ngx/paperless-ngx/pull/14682))
### Maintenance
- Chore(deps): Bump the actions group across 1 directory with 17 updates @[dependabot[bot]](https://github.com/apps/dependabot) ([#14691](https://github.com/paperless-ngx/paperless-ngx/pull/14691))
### Dependencies
<details>
<summary>4 changes</summary>
- Chore(deps): Bump the pre-commit-dependencies group with 4 updates @[dependabot[bot]](https://github.com/apps/dependabot) ([#14636](https://github.com/paperless-ngx/paperless-ngx/pull/14636))
- Chore(deps): Bump the actions group across 1 directory with 17 updates @[dependabot[bot]](https://github.com/apps/dependabot) ([#14691](https://github.com/paperless-ngx/paperless-ngx/pull/14691))
- docker(deps): bump astral-sh/uv from 0.13.5-python3.14-trixie-slim to 0.13.7-python3.14-trixie-slim @[dependabot[bot]](https://github.com/apps/dependabot) ([#14635](https://github.com/paperless-ngx/paperless-ngx/pull/14635))
- chore(deps): bump django-allauth to 65.21.1 [@stumpylog](https://github.com/stumpylog) ([#14672](https://github.com/paperless-ngx/paperless-ngx/pull/14672))
</details>
### All App Changes
<details>
<summary>32 changes</summary>
- Chorehancement: move all bulk operations to async hand-off [@shamoon](https://github.com/shamoon) ([#14773](https://github.com/paperless-ngx/paperless-ngx/pull/14773))
- Fix: use a constant password mask and consolidate the no-change check [@stumpylog](https://github.com/stumpylog) ([#14784](https://github.com/paperless-ngx/paperless-ngx/pull/14784))
- Fix: use a PostgreSQL-compatible regex for scheduled workflow filename filters [@stumpylog](https://github.com/stumpylog) ([#14783](https://github.com/paperless-ngx/paperless-ngx/pull/14783))
- Fix: get users for saved view create dialog [@shamoon](https://github.com/shamoon) ([#14769](https://github.com/paperless-ngx/paperless-ngx/pull/14769))
- Enhancement: render document notes as markdown [@zwif](https://github.com/zwif) ([#14626](https://github.com/paperless-ngx/paperless-ngx/pull/14626))
- Feature: manual workflow trigger [@shamoon](https://github.com/shamoon) ([#14704](https://github.com/paperless-ngx/paperless-ngx/pull/14704))
- Fix: Allow AI chat filters over more than 32700 documents [@stumpylog](https://github.com/stumpylog) ([#14754](https://github.com/paperless-ngx/paperless-ngx/pull/14754))
- Fix: Enforce CRLF when writing an email message to the file [@stumpylog](https://github.com/stumpylog) ([#14745](https://github.com/paperless-ngx/paperless-ngx/pull/14745))
- Enhancement: make bulk metadata operations run async [@shamoon](https://github.com/shamoon) ([#14711](https://github.com/paperless-ngx/paperless-ngx/pull/14711))
- Performance: load a cached Hugging Face embedding model without contacting the Hub [@stumpylog](https://github.com/stumpylog) ([#14728](https://github.com/paperless-ngx/paperless-ngx/pull/14728))
- Fix: restore image without archive version thumbnail generation [@shamoon](https://github.com/shamoon) ([#14741](https://github.com/paperless-ngx/paperless-ngx/pull/14741))
- Fix: Blacklist certain OCR user arguments that paperless itself uses [@stumpylog](https://github.com/stumpylog) ([#14734](https://github.com/paperless-ngx/paperless-ngx/pull/14734))
- Fix: remove redundant ui task text filters [@shamoon](https://github.com/shamoon) ([#14736](https://github.com/paperless-ngx/paperless-ngx/pull/14736))
- Fix: fix allowed\_hosts default [@shamoon](https://github.com/shamoon) ([#14735](https://github.com/paperless-ngx/paperless-ngx/pull/14735))
- Fixhancement: add automatic clearing of sessions [@shamoon](https://github.com/shamoon) ([#14722](https://github.com/paperless-ngx/paperless-ngx/pull/14722))
- Tweak: chat reference tweaks [@shamoon](https://github.com/shamoon) ([#14723](https://github.com/paperless-ngx/paperless-ngx/pull/14723))
- Fix: two dropdown-related UI fixes [@shamoon](https://github.com/shamoon) ([#14717](https://github.com/paperless-ngx/paperless-ngx/pull/14717))
- Fix: use filtered documents for statistics ASN max [@shamoon](https://github.com/shamoon) ([#14710](https://github.com/paperless-ngx/paperless-ngx/pull/14710))
- Fix: make config view permissions consistent [@shamoon](https://github.com/shamoon) ([#14709](https://github.com/paperless-ngx/paperless-ngx/pull/14709))
- Performance: stop embedding every document ID in selection\_data queries [@stumpylog](https://github.com/stumpylog) ([#14597](https://github.com/paperless-ngx/paperless-ngx/pull/14597))
- Performance: replace django-treenode with adjacency-list tags [@shamoon](https://github.com/shamoon) ([#14612](https://github.com/paperless-ngx/paperless-ngx/pull/14612))
- Performance: use small vec0 chunks and larger pages for the LLM index [@stumpylog](https://github.com/stumpylog) ([#14678](https://github.com/paperless-ngx/paperless-ngx/pull/14678))
- Fix: release the LLM index read lock before streaming the chat answer [@stumpylog](https://github.com/stumpylog) ([#14674](https://github.com/paperless-ngx/paperless-ngx/pull/14674))
- Fix: retrieve once per document chat question [@stumpylog](https://github.com/stumpylog) ([#14673](https://github.com/paperless-ngx/paperless-ngx/pull/14673))
- Fixhancement: rework saved view widget queue processing to expected [@shamoon](https://github.com/shamoon) ([#14699](https://github.com/paperless-ngx/paperless-ngx/pull/14699))
- Fix: apply PDF/A conversion once for mail archives [@shamoon](https://github.com/shamoon) ([#14698](https://github.com/paperless-ngx/paperless-ngx/pull/14698))
- Fix default MFA trusted AMR setting [@shamoon](https://github.com/shamoon) ([#14693](https://github.com/paperless-ngx/paperless-ngx/pull/14693))
- Fix: check descendant tag permissions on delete [@shamoon](https://github.com/shamoon) ([#14687](https://github.com/paperless-ngx/paperless-ngx/pull/14687))
- Fix: lock rows for first-user signup [@shamoon](https://github.com/shamoon) ([#14688](https://github.com/paperless-ngx/paperless-ngx/pull/14688))
- Fix: prevent ID badge wrapping under title on mobile [@shamoon](https://github.com/shamoon) ([#14682](https://github.com/paperless-ngx/paperless-ngx/pull/14682))
- chore(deps): bump django-allauth to 65.21.1 [@stumpylog](https://github.com/stumpylog) ([#14672](https://github.com/paperless-ngx/paperless-ngx/pull/14672))
- chore(deps): bump dev tooling [@stumpylog](https://github.com/stumpylog) ([#14671](https://github.com/paperless-ngx/paperless-ngx/pull/14671))
</details>
## paperless-ngx 3.4.2
### Bug Fixes
+3 -4
View File
@@ -1,6 +1,6 @@
[project]
name = "paperless-ngx"
version = "3.5.1"
version = "3.4.2"
description = """\
A community-supported supercharged document management system: scan, index and archive all your physical documents\
"""
@@ -151,9 +151,8 @@ typing = [
required-version = ">=0.9.0"
prerelease = "disallow"
environments = [
"sys_platform == 'darwin' and platform_machine == 'arm64' and platform_python_implementation == 'CPython'",
"sys_platform == 'linux' and platform_machine == 'aarch64' and platform_python_implementation == 'CPython'",
"sys_platform == 'linux' and platform_machine == 'x86_64' and platform_python_implementation == 'CPython'",
"sys_platform == 'darwin'",
"sys_platform == 'linux'",
]
package = false
[tool.uv.sources]
+249 -490
View File
File diff suppressed because it is too large. Load diff
+1 -1
View File
@@ -1,6 +1,6 @@
{
"name": "paperless-ngx-ui",
"version": "3.5.1",
"version": "3.4.2",
"scripts": {
"preinstall": "npx only-allow pnpm",
"ng": "ng",
@@ -227,12 +227,6 @@
</div>
</div>
<div class="row">
<div class="col">
<pngx-input-check i18n-title title="Show nested tags with their parents" i18n-hint hint="Child tags are shown together with their parent tags, instead of separately." formControlName="tagsShowHierarchy"></pngx-input-check>
</div>
</div>
<h5 class="mt-3" i18n>Document editing</h5>
<div class="row">
<div class="col">
@@ -305,7 +305,7 @@ describe('SettingsComponent', () => {
expect(toastErrorSpy).toHaveBeenCalled()
expect(storeSpy).toHaveBeenCalled()
expect(appearanceSettingsSpy).not.toHaveBeenCalled()
expect(setSpy).toHaveBeenCalledTimes(34)
expect(setSpy).toHaveBeenCalledTimes(33)
expect(setSpy).toHaveBeenCalledWith(SETTINGS_KEYS.SIDEBAR_HIDDEN_ITEMS, [
HideableSidebarItemID.Workflows,
])
@@ -163,7 +163,6 @@ export class SettingsComponent
bulkEditConfirmationDialogs: new FormControl(null),
bulkEditApplyOnClose: new FormControl(null),
documentListItemPerPage: new FormControl(null),
tagsShowHierarchy: new FormControl(null),
slimSidebarEnabled: new FormControl(null),
sidebarHiddenItems: new FormControl<HideableSidebarItemID[]>([]),
darkModeUseSystem: new FormControl(null),
@@ -343,7 +342,6 @@ export class SettingsComponent
bulkEditApplyOnClose: this.settings.get(
SETTINGS_KEYS.BULK_EDIT_APPLY_ON_CLOSE
),
tagsShowHierarchy: this.settings.get(SETTINGS_KEYS.TAGS_SHOW_HIERARCHY),
documentListItemPerPage: this.settings.get(
SETTINGS_KEYS.DOCUMENT_LIST_SIZE
),
@@ -609,10 +607,6 @@ export class SettingsComponent
SETTINGS_KEYS.DEFAULT_PERMS_EDIT_GROUPS,
this.settingsForm.value.defaultPermsEditGroups
)
this.settings.set(
SETTINGS_KEYS.TAGS_SHOW_HIERARCHY,
this.settingsForm.value.tagsShowHierarchy
)
this.settings.set(
SETTINGS_KEYS.DOCUMENT_EDITING_REMOVE_INBOX_TAGS,
this.settingsForm.value.documentEditingRemoveInboxTags
@@ -186,9 +186,7 @@ export class AppFrameComponent
)
.subscribe((result) => {
this.documentList.reload(null, false)
if (!this.documentList.allSelected) {
this.documentList.reduceSelectionToFilter()
}
this.documentList.reduceSelectionToFilter()
// maybe someday we can optimize to only refresh affected documents
this.openDocumentsService.getOpenDocuments().forEach((doc) => {
this.openDocumentsService.refreshDocument(doc.id)
@@ -188,62 +188,6 @@ describe('ChatComponent', () => {
expect(component.loading()).toBe(false)
})
it('should reveal a complete answer within one second', () => {
component.input.set('Hello')
component.sendMessage()
const answer = 'A'.repeat(660)
mockStream$.next(answer)
mockStream$.complete()
expect(component.messages()[1].content.length).toBeGreaterThan(0)
expect(component.messages()[1].content.length).toBeLessThan(answer.length)
expect(component.loading()).toBe(true)
jest.advanceTimersByTime(800)
expect(component.messages()[1].content).toBe(answer)
expect(component.loading()).toBe(false)
})
it('should give a long answer more time to reveal', () => {
component.input.set('Hello')
component.sendMessage()
const answer = 'A'.repeat(3000)
mockStream$.next(answer)
mockStream$.complete()
jest.advanceTimersByTime(800)
expect(component.messages()[1].content.length).toBeLessThan(answer.length)
expect(component.loading()).toBe(true)
jest.advanceTimersByTime(2500)
expect(component.messages()[1].content).toBe(answer)
expect(component.loading()).toBe(false)
})
it('should reveal fast incoming chunks without making them pop in', () => {
component.input.set('Hello')
component.sendMessage()
let answer = ''
for (let i = 0; i < 20; i++) {
answer += 'A'.repeat(100)
const visibleBefore = component.messages()[1].content.length
mockStream$.next(answer)
jest.advanceTimersByTime(10)
expect(
component.messages()[1].content.length - visibleBefore
).toBeLessThan(100)
}
expect(component.messages()[1].content.length).toBeLessThan(answer.length)
mockStream$.complete()
jest.advanceTimersByTime(5200)
expect(component.messages()[1].content).toBe(answer)
expect(component.loading()).toBe(false)
})
it('should show a slow response message while waiting', () => {
component.input.set('Hello')
component.sendMessage()
@@ -21,9 +21,6 @@ import {
import { pngxPopperOptions } from 'src/app/utils/popper-options'
const SLOW_RESPONSE_DELAY_MS = 8000
const TYPEWRITER_MIN_DURATION_MS = 750
const TYPEWRITER_MAX_DURATION_MS = 3000
const TYPEWRITER_INTERVAL_MS = 16
@Component({
selector: 'pngx-chat',
@@ -76,7 +73,6 @@ export class ChatComponent implements OnInit {
private typewriterBuffer: string[] = []
private typewriterActive = false
private typewriterTimeout: ReturnType<typeof setTimeout>
private typewriterDeadline = 0
private streamSubscription: Subscription
private streamingMessage: ChatMessage
private streamFinished = false
@@ -138,7 +134,7 @@ export class ChatComponent implements OnInit {
if (nextResponse.content.length < lastVisibleContent.length) {
this.resetTypewriter(assistantMessage, nextResponse.content)
lastVisibleContent = nextResponse.content
} else if (nextResponse.content.length > lastVisibleContent.length) {
} else {
const visibleDelta = nextResponse.content.substring(
lastVisibleContent.length
)
@@ -218,12 +214,6 @@ export class ChatComponent implements OnInit {
if (!chunk) return
this.typewriterBuffer.push(...chunk.split(''))
// Longer bursts need more time to remain readable as the chat scrolls.
const revealDuration = Math.min(
TYPEWRITER_MAX_DURATION_MS,
Math.max(TYPEWRITER_MIN_DURATION_MS, this.typewriterBuffer.length)
)
this.typewriterDeadline = Date.now() + revealDuration
if (!this.typewriterActive) {
this.typewriterActive = true
@@ -232,29 +222,19 @@ export class ChatComponent implements OnInit {
}
playTypewriter(message: ChatMessage): void {
// Spread the queued text across the updates left before the deadline
const remainingUpdates = Math.max(
1,
Math.ceil((this.typewriterDeadline - Date.now()) / TYPEWRITER_INTERVAL_MS)
)
const charactersToShow = Math.ceil(
this.typewriterBuffer.length / remainingUpdates
)
message.content += this.typewriterBuffer
.splice(0, charactersToShow)
.join('')
this.notifyMessagesChanged()
this.scrollToBottom()
if (this.typewriterBuffer.length === 0) {
this.typewriterActive = false
if (this.streamFinished) this.finishStreaming(message)
} else {
this.typewriterTimeout = setTimeout(
() => this.playTypewriter(message),
TYPEWRITER_INTERVAL_MS
)
return
}
const nextChar = this.typewriterBuffer.shift()
message.content += nextChar
this.notifyMessagesChanged()
this.scrollToBottom()
// 10ms per character
this.typewriterTimeout = setTimeout(() => this.playTypewriter(message), 10)
}
private notifyMessagesChanged(): void {
@@ -1,90 +0,0 @@
import { provideHttpClient } from '@angular/common/http'
import { provideHttpClientTesting } from '@angular/common/http/testing'
import { Component } from '@angular/core'
import { ComponentFixture, TestBed } from '@angular/core/testing'
import { FormControl, ReactiveFormsModule } from '@angular/forms'
import { NumberComponent } from './number/number.component'
import { TextComponent } from './text/text.component'
import { TextAreaComponent } from './textarea/textarea.component'
import { UrlComponent } from './url/url.component'
@Component({
imports: [
ReactiveFormsModule,
TextComponent,
NumberComponent,
TextAreaComponent,
UrlComponent,
],
template: `
<pngx-input-text [formControl]="control" />
<pngx-input-number [formControl]="control" [showAdd]="false" />
<pngx-input-textarea [formControl]="control" />
<pngx-input-url [formControl]="control" />
`,
})
class InputEventsHostComponent {
control = new FormControl<string | number | null>(null)
}
describe.each([
['text', 'pngx-input-text input', 'Edited text', 'Edited text'],
['number', 'pngx-input-number input', '42', 42],
['textarea', 'pngx-input-textarea textarea', 'Edited\ntext', 'Edited\ntext'],
['url', 'pngx-input-url input', 'https://example.com', 'https://example.com'],
])(
'%s input events',
(_type, selector: string, text: string, value: string | number) => {
let fixture: ComponentFixture<InputEventsHostComponent>
let control: FormControl<string | number | null>
let input: HTMLInputElement | HTMLTextAreaElement
beforeEach(() => {
TestBed.configureTestingModule({
imports: [InputEventsHostComponent],
providers: [provideHttpClient(), provideHttpClientTesting()],
})
fixture = TestBed.createComponent(InputEventsHostComponent)
control = fixture.componentInstance.control
})
async function render() {
fixture.detectChanges()
await fixture.whenStable()
input = fixture.nativeElement.querySelector(selector)
}
it('updates the form while typing and marks it touched only on blur', async () => {
await render()
const changed = jest.fn()
control.valueChanges.subscribe(changed)
input.value = text
input.dispatchEvent(new Event('input'))
expect(control.value).toBe(value)
expect(control.dirty).toBe(true)
expect(control.touched).toBe(false)
input.dispatchEvent(new Event('change'))
input.dispatchEvent(new Event('blur'))
expect(control.touched).toBe(true)
expect(changed).toHaveBeenCalledTimes(1)
})
it('honors an explicit updateOn blur strategy', async () => {
control = new FormControl<string | number | null>(null, {
updateOn: 'blur',
})
fixture.componentInstance.control = control
await render()
input.value = text
input.dispatchEvent(new Event('input'))
expect(control.value).toBeNull()
expect(control.pristine).toBe(true)
input.dispatchEvent(new Event('blur'))
expect(control.value).toBe(value)
expect(control.dirty).toBe(true)
expect(control.touched).toBe(true)
})
}
)
@@ -12,7 +12,7 @@
</div>
<div class="position-relative" [class.col-md-9]="horizontal">
<div class="input-group" [class.is-invalid]="error">
<input #inputField type="number" class="form-control" [step]="step" [id]="inputId" [(ngModel)]="value" [ngModelOptions]="{standalone: true}" (ngModelChange)="onChange($event)" (blur)="onTouched()" [class.is-invalid]="error" [disabled]="disabled">
<input #inputField type="number" class="form-control" [step]="step" [id]="inputId" [(ngModel)]="value" [ngModelOptions]="{standalone: true}" (change)="onChange(value)" [class.is-invalid]="error" [disabled]="disabled">
@if (showAdd) {
<button class="btn btn-outline-secondary" type="button" id="button-addon1" (click)="nextAsn()" [disabled]="disabled">+1</button>
}
@@ -17,21 +17,16 @@
[searchFn]="searchFn"
addTagText="Add tag"
i18n-addTagText
[keyDownFn]="onKeyDown"
(add)="onAdd($event)"
(change)="onChange(value)">
<ng-template ng-label-tmp let-item="item">
@if (isImplied(item.id)) {
<span class="implied-tag"></span>
} @else {
<button class="tag-wrap btn p-0 d-flex align-items-center mw-100" (click)="removeBadge(item.id)" (mousedown)="$event.stopImmediatePropagation()" type="button" title="Remove tag" i18n-title>
<pngx-bs-icon name="x" style="margin-inline-end: 1px;"></pngx-bs-icon>
@if (item.id && tags) {
<pngx-tag class="mw-100 overflow-hidden" [tag]="getTag(item.id)" [split]="preserveHierarchy" (click)="removeTag(item.id); $event.stopPropagation()" (parentClick)="removeTag($event)"></pngx-tag>
}
</button>
}
<button class="tag-wrap btn p-0 d-flex align-items-center" (click)="removeTag(item.id)" (mousedown)="$event.stopImmediatePropagation()" type="button" title="Remove tag" i18n-title>
<pngx-bs-icon name="x" style="margin-inline-end: 1px;"></pngx-bs-icon>
@if (item.id && tags) {
<pngx-tag style="background-color: none;" [tag]="getTag(item.id)"></pngx-tag>
}
</button>
</ng-template>
<ng-template ng-option-tmp let-item="item" let-index="index" let-search="searchTerm">
<div class="tag-option-row d-flex align-items-center" [class.w-auto]="!getTag(item.id)?.parent">
@@ -7,11 +7,6 @@
font-size: 1rem;
}
// ancestors of another selected tag render inside that tag's badge
::ng-deep .paperless-input-tags .ng-value:has(> .implied-tag) {
display: none !important;
}
.paperless-input-select.disabled {
.input-group {
cursor: not-allowed;
@@ -278,59 +278,4 @@ describe('TagsComponent', () => {
component.tags = [lone]
expect(component.getParentChain(5)).toEqual([])
})
describe('hierarchy', () => {
const root: Tag = { id: 1, name: 'root' }
const mid: Tag = { id: 2, name: 'mid', parent: 1 }
const leaf: Tag = { id: 3, name: 'leaf', parent: 2 }
const other: Tag = { id: 4, name: 'other' }
beforeEach(() => {
component.tags = [root, mid, leaf, other]
component.value = [3, 2, 1, 4]
})
it('should treat ancestors of another selected tag as implied', () => {
expect([1, 2, 3, 4].map((id) => component.isImplied(id))).toEqual([
true,
true,
false,
false,
])
component.preserveHierarchy = false
expect(component.isImplied(1)).toBe(false)
})
it('should remove a whole badge, keeping ancestors other tags still need', () => {
component.removeBadge(3)
expect(component.value).toEqual([4])
const sibling: Tag = { id: 5, name: 'sibling', parent: 1 }
component.tags = [root, mid, leaf, other, sibling]
component.value = [3, 2, 1, 5, 4]
component.removeBadge(3)
expect(component.value).toEqual([1, 5, 4])
})
it('should remove only the tag itself from a badge when hierarchy is not shown', () => {
component.preserveHierarchy = false
component.removeBadge(3)
expect(component.value).toEqual([2, 1, 4])
})
it('should remove the last visible tag on backspace, not a hidden ancestor', () => {
const backspace = new KeyboardEvent('keydown', { key: 'Backspace' })
expect(component.onKeyDown(backspace)).toBe(false)
expect(component.value).toEqual([3, 2, 1])
expect(component.onKeyDown(backspace)).toBe(false)
expect(component.value).toEqual([2, 1])
// nothing hidden: leave it to ng-select
component.value = [4]
expect(component.onKeyDown(backspace)).toBe(true)
expect(
component.onKeyDown(new KeyboardEvent('keydown', { key: 'a' }))
).toBe(true)
})
})
})
@@ -21,13 +21,7 @@ import { NgSelectComponent, NgSelectModule } from '@ng-select/ng-select'
import { first, firstValueFrom, tap } from 'rxjs'
import { BootstrapIconComponent } from 'src/app/components/common/bootstrap-icon/bootstrap-icon.component'
import { Tag } from 'src/app/data/tag'
import { SETTINGS_KEYS } from 'src/app/data/ui-settings'
import { TagService } from 'src/app/services/rest/tag.service'
import { SettingsService } from 'src/app/services/settings.service'
import {
collapseImpliedAncestors,
getTagAncestors,
} from 'src/app/utils/collapse-tags'
import { matchesSearchText } from 'src/app/utils/text-search'
import { EditDialogMode } from '../../edit-dialog/edit-dialog.component'
import { TagEditDialogComponent } from '../../edit-dialog/tag-edit-dialog/tag-edit-dialog.component'
@@ -56,7 +50,6 @@ import { TagComponent } from '../../tag/tag.component'
export class TagsComponent implements OnInit, ControlValueAccessor {
private tagService = inject(TagService)
private modalService = inject(NgbModal)
private readonly settingsService = inject(SettingsService)
private readonly changeDetector = inject(ChangeDetectorRef)
constructor() {
@@ -138,33 +131,6 @@ export class TagsComponent implements OnInit, ControlValueAccessor {
term
)
/**
* Selected tags that are ancestors of another selected tag which are shown
* as segments of their descendant's badge.
*/
isImplied(id: number): boolean {
if (!this.showsHierarchy || !this.tags || !this.value) return false
return !collapseImpliedAncestors(this.value, this.tagsById).includes(id)
}
private get showsHierarchy(): boolean {
return (
this.preserveHierarchy &&
!!this.settingsService.get(SETTINGS_KEYS.TAGS_SHOW_HIERARCHY)
)
}
/**
* Remove the last visible tag (in case its actually a hidden ancestor)
*/
readonly onKeyDown = (event: KeyboardEvent): boolean => {
if (event.key !== 'Backspace' || this.select?.searchTerm) return true
const visible = this.value?.filter((id) => !this.isImplied(id)) ?? []
if (visible.length === (this.value?.length ?? 0)) return true
this.removeTag(visible.at(-1))
return false
}
getTag(id: number) {
if (this.tags) {
return this.tags.find((tag) => tag.id == id)
@@ -173,26 +139,6 @@ export class TagsComponent implements OnInit, ControlValueAccessor {
}
}
/**
* Remove a whole badge i.e. the tag plus any ancestors that no other selected
* tag still needs.
*/
removeBadge(tagID: number) {
if (this.disabled) return
if (!this.showsHierarchy) return this.removeTag(tagID)
const ancestors = new Set(this.getParentChain(tagID).map((t) => t.id))
const needed = new Set(
this.value
.filter((id) => id !== tagID && !ancestors.has(id))
.flatMap((id) => this.getParentChain(id).map((t) => t.id))
)
this.value = this.value.filter(
(id) => id !== tagID && (!ancestors.has(id) || needed.has(id))
)
this.onChange(this.value)
}
removeTag(tagID: number) {
if (this.disabled) return
@@ -289,17 +235,18 @@ export class TagsComponent implements OnInit, ControlValueAccessor {
}
getParentChain(id: number): Tag[] {
return getTagAncestors(id, this.tagsById)
}
private tagsByIdSource: Tag[]
private tagsByIdCache: Map<number, Tag>
private get tagsById(): Map<number, Tag> {
if (this.tagsByIdSource !== this.tags) {
this.tagsByIdSource = this.tags
this.tagsByIdCache = new Map(this.tags?.map((t) => [t.id, t]))
// Returns ancestors from root → immediate parent for a tag id
const chain: Tag[] = []
let current = this.getTag(id)
const guard = new Set<number>()
while (current?.parent) {
if (guard.has(current.parent)) break
guard.add(current.parent)
const parent = this.getTag(current.parent)
if (!parent) break
chain.unshift(parent)
current = parent
}
return this.tagsByIdCache
return chain
}
}
@@ -11,7 +11,7 @@
}
</div>
<div class="position-relative" [class.col-md-9]="horizontal">
<input #inputField type="text" class="form-control" [class.is-invalid]="error" [id]="inputId" [(ngModel)]="value" [ngModelOptions]="{standalone: true}" (ngModelChange)="onChange($event)" (blur)="onTouched()" [disabled]="disabled" [autocomplete]="autocomplete" [placeholder]="placeholder">
<input #inputField type="text" class="form-control" [class.is-invalid]="error" [id]="inputId" [(ngModel)]="value" [ngModelOptions]="{standalone: true}" (change)="onChange(value)" [disabled]="disabled" [autocomplete]="autocomplete" [placeholder]="placeholder">
@if (hint) {
<small class="form-text text-muted" [innerHTML]="hint"></small>
}
@@ -18,8 +18,7 @@
[class.font-monospace]="monospace"
[(ngModel)]="value"
[ngModelOptions]="{standalone: true}"
(ngModelChange)="onChange($event)"
(blur)="onTouched()"
(change)="onChange(value)"
[disabled]="disabled"
[placeholder]="placeholder"
rows="4">
@@ -10,7 +10,7 @@
</div>
<div [class.col-md-9]="horizontal">
<div class="input-group" [class.is-invalid]="error">
<input #inputField type="url" class="form-control" [class.is-invalid]="error" placeholder="https://" [id]="inputId" [(ngModel)]="value" [ngModelOptions]="{standalone: true}" (ngModelChange)="onChange($event)" (blur)="onTouched()" [disabled]="disabled">
<input #inputField type="url" class="form-control" [class.is-invalid]="error" placeholder="https://" [id]="inputId" [(ngModel)]="value" [ngModelOptions]="{standalone: true}" (change)="onChange(value)" [disabled]="disabled">
<a class="btn btn-outline-secondary rounded-end" title="Open link" i18n-title [href]="value" target="_blank">
<pngx-bs-icon width="1.2em" height="1.2em" name="box-arrow-up-right"></pngx-bs-icon>
</a>
@@ -1,18 +1,13 @@
@if (tag()) {
@if (ancestorSegments.length) {
<span #pill class="split-pill d-inline-flex mw-100" [title]="path()" (mouseleave)="setFocus(pill, null)">
@for (ancestor of ancestorSegments; track ancestor.id; let first = $first) {
<span #segment class="badge split-parent" [class.rounded-end-0]="first" [class.rounded-0]="!first" [style.background]="ancestor.color" [style.color]="ancestor.text_color" (mouseenter)="setFocus(pill, segment)" (click)="onParentClick($event, ancestor.id)">{{ancestor.name}}</span>
}
<span #leaf class="badge rounded-start-0 split-child" [style.background]="tag().color" [style.color]="tag().text_color" (mouseenter)="setFocus(pill, leaf)">{{tag().name}}</span>
</span>
} @else {
@if (!clickable()) {
<span class="badge" [style.background]="tag().color" [style.color]="tag().text_color">{{tag().name}}</span>
}
@if (clickable()) {
<a [title]="linkTitle()" class="badge" [style.background]="tag().color" [style.color]="tag().text_color">{{tag().name}}</a>
}
@if (showParents() && tag().parent) {
<pngx-tag [tagID]="tag().parent" [clickable]="clickable()" [linkTitle]="linkTitle()"></pngx-tag>
&nbsp;&gt;&nbsp;
}
@if (!clickable()) {
<span class="badge" [style.background]="tag().color" [style.color]="tag().text_color">{{tag().name}}</span>
}
@if (clickable()) {
<a [title]="linkTitle()" class="badge" [style.background]="tag().color" [style.color]="tag().text_color">{{tag().name}}</a>
}
} @else if (loading) {
<span class="placeholder-glow">
@@ -11,45 +11,3 @@ a, span {
opacity: .5;
font-style: italic;
}
.split-parent,
.split-child {
overflow: hidden;
white-space: nowrap;
transition: color .2s ease;
}
.split-parent {
min-width: 1.5em;
max-width: 10em;
padding-inline: .4em;
text-overflow: clip;
&:nth-last-child(2) { flex-shrink: 1e12; }
&:nth-last-child(3) { flex-shrink: 1e18; }
&:nth-last-child(4) { flex-shrink: 1e24; }
&:nth-last-child(5) { flex-shrink: 1e30; }
&.collapsed {
color: transparent !important;
}
}
.split-child {
flex-shrink: 1e6;
min-width: 2em;
text-overflow: ellipsis;
}
// Hovered segment gets priority, other parents can shrink to a sliver
.split-pill {
> .split-parent.focused,
> .split-child.focused {
flex-shrink: 1;
text-overflow: ellipsis;
}
&:has(> .focused) > .split-parent:not(.focused) {
min-width: 0;
}
}
@@ -6,7 +6,6 @@ import { of } from 'rxjs'
import { Tag } from 'src/app/data/tag'
import { PermissionsService } from 'src/app/services/permissions.service'
import { TagService } from 'src/app/services/rest/tag.service'
import { SettingsService } from 'src/app/services/settings.service'
import { TagComponent } from './tag.component'
const tag: Tag = {
@@ -70,88 +69,4 @@ describe('TagComponent', () => {
expect(getCachedSpy).toHaveBeenCalledWith(1)
expect(component.tag()).toEqual(tag)
})
describe('split badge', () => {
const root: Tag = { id: 1, name: 'Root', color: '#000000' }
const mid: Tag = { id: 2, name: 'Mid', color: '#111111', parent: 1 }
const leaf: Tag = { id: 3, name: 'Leaf', color: '#222222', parent: 2 }
const segments = (): HTMLElement[] =>
fixture.debugElement
.queryAll(By.css('.split-pill > .badge'))
.map((d) => d.nativeElement)
beforeEach(() => {
jest.spyOn(permissionsService, 'currentUserCan').mockReturnValue(true)
jest.spyOn(tagService, 'getCached').mockReturnValue(of(leaf))
jest
.spyOn(tagService, 'listAll')
.mockReturnValue(
of({ count: 3, all: [1, 2, 3], results: [root, mid, leaf] })
)
fixture.componentRef.setInput('tagID', 3)
fixture.componentRef.setInput('split', true)
fixture.detectChanges()
})
it('should render ancestors as segments with the full path as title', () => {
expect(segments().map((s) => s.textContent.trim())).toEqual([
'Root',
'Mid',
'Leaf',
])
expect(
fixture.debugElement.query(By.css('.split-pill')).nativeElement.title
).toEqual('Root › Mid › Leaf')
})
it('should render a plain badge when the setting is off', () => {
jest.spyOn(TestBed.inject(SettingsService), 'get').mockReturnValue(false)
fixture.componentRef.setInput('split', false)
fixture.detectChanges()
fixture.componentRef.setInput('split', true)
fixture.detectChanges()
expect(fixture.debugElement.query(By.css('.split-pill'))).toBeNull()
expect(component.ancestorSegments).toEqual([])
})
it('should emit the ancestor id when an ancestor segment is clicked', () => {
const emitSpy = jest.spyOn(component.parentClick, 'emit')
const event = new MouseEvent('click', { bubbles: true })
const stopSpy = jest.spyOn(event, 'stopImmediatePropagation')
segments()[1].dispatchEvent(event)
expect(emitSpy).toHaveBeenCalledWith(2)
expect(stopSpy).toHaveBeenCalled()
})
it('should focus the hovered segment and animate width changes', () => {
const [rootSeg, midSeg, leafSeg] = segments()
const pill = rootSeg.parentElement
// before / after widths per segment, as measured by setFocus
const widths = new Map<HTMLElement, number[]>([
[rootSeg, [20, 4]],
[midSeg, [20, 60]],
[leafSeg, [40, 40]],
])
const animate = jest.fn()
for (const [seg, [before, after]] of widths) {
seg.getAnimations = jest.fn(() => [])
seg.animate = animate
jest
.spyOn(seg, 'getBoundingClientRect')
.mockReturnValueOnce({ width: before } as DOMRect)
.mockReturnValueOnce({ width: after } as DOMRect)
}
midSeg.dispatchEvent(new MouseEvent('mouseenter'))
expect(midSeg.classList).toContain('focused')
expect(rootSeg.classList).not.toContain('focused')
// only the segments whose width changed are animated
expect(animate).toHaveBeenCalledTimes(2)
pill.dispatchEvent(new MouseEvent('mouseleave'))
expect(segments().some((s) => s.classList.contains('focused'))).toBe(
false
)
})
})
})
@@ -1,27 +1,11 @@
import {
Component,
computed,
effect,
inject,
input,
model,
output,
signal,
} from '@angular/core'
import { Component, effect, inject, input, model } from '@angular/core'
import { Tag } from 'src/app/data/tag'
import { SETTINGS_KEYS } from 'src/app/data/ui-settings'
import {
PermissionAction,
PermissionsService,
PermissionType,
} from 'src/app/services/permissions.service'
import { TagService } from 'src/app/services/rest/tag.service'
import { SettingsService } from 'src/app/services/settings.service'
import { getTagAncestors } from 'src/app/utils/collapse-tags'
const FOCUS_ANIMATION_MS = 200
// matches the .split-parent min-width in tag.component.scss
const COLLAPSE_BELOW_EM = 1.5
@Component({
selector: 'pngx-tag',
@@ -29,9 +13,8 @@ const COLLAPSE_BELOW_EM = 1.5
styleUrls: ['./tag.component.scss'],
})
export class TagComponent {
private readonly permissionsService = inject(PermissionsService)
private readonly tagService = inject(TagService)
private readonly settingsService = inject(SettingsService)
private permissionsService = inject(PermissionsService)
private tagService = inject(TagService)
readonly tag = model<Tag>(null)
readonly tagID = input<number>(undefined)
@@ -40,101 +23,27 @@ export class TagComponent {
transform: (value: boolean | string) =>
value === '' || value === true || value === 'true',
})
readonly split = input(false)
readonly parentClick = output<number>()
private readonly ancestors = signal<Tag[]>([])
public get ancestorSegments(): Tag[] {
return this.split() &&
this.settingsService.get(SETTINGS_KEYS.TAGS_SHOW_HIERARCHY)
? this.ancestors()
: []
}
public readonly path = computed(() =>
[...this.ancestors(), this.tag()]
.filter(Boolean)
.map((t) => t.name)
.join(' › ')
)
readonly showParents = input(false)
constructor() {
effect(() => {
const tagID = this.tagID()
if (tagID && this.canViewTags) {
this.tagService.getCached(tagID).subscribe((tag) => {
this.tag.set(tag)
})
}
})
// ancestors for split badges, from either tagID or a passed-in tag
effect(() => {
const id = this.tagID() || this.tag()?.id
if (!this.split() || !id || !this.canViewTags) {
this.ancestors.set([])
return
}
this.tagService.listAll().subscribe((tags) => {
this.ancestors.set(
getTagAncestors(
id,
new Map<number, Tag>(tags.results.map((t) => [t.id, t]))
if (tagID) {
if (
this.permissionsService.currentUserCan(
PermissionAction.View,
PermissionType.Tag
)
)
})
) {
this.tagService.getCached(tagID).subscribe((tag) => {
this.tag.set(tag)
})
}
}
})
}
private get canViewTags(): boolean {
return this.permissionsService.currentUserCan(
PermissionAction.View,
PermissionType.Tag
)
}
public get loading(): boolean {
return this.tagService.loading
}
/**
* Gives the hovered segment priority (.focused class, see scss)
* and animates each from old width to new
*/
setFocus(segment: HTMLElement, focused: HTMLElement | null) {
const segments = Array.from(segment.children) as HTMLElement[]
const before = segments.map((s) => s.getBoundingClientRect().width)
for (const segment of segments) {
segment.getAnimations().forEach((a) => a.cancel())
segment.classList.toggle('focused', segment === focused)
}
const after = segments.map((s) => s.getBoundingClientRect().width)
segments.forEach((segment, i) => {
const em = Number.parseFloat(getComputedStyle(segment).fontSize)
segment.classList.toggle(
'collapsed',
!!focused &&
segment.classList.contains('split-parent') &&
after[i] < COLLAPSE_BELOW_EM * em
)
if (Math.abs(after[i] - before[i]) < 0.5) return
segment.animate(
[
{ flex: `0 0 ${before[i]}px`, minWidth: 0 },
{ flex: `0 0 ${after[i]}px`, minWidth: 0 },
],
{ duration: FOCUS_ANIMATION_MS, easing: 'ease' }
)
})
}
onParentClick(event: MouseEvent, id: number) {
event.preventDefault()
event.stopImmediatePropagation()
this.parentClick.emit(id)
}
}
@@ -76,8 +76,8 @@
}
}
@case (DisplayField.TAGS) {
@for (tagID of doc.tags | collapseTags | async; track tagID) {
<pngx-tag [tagID]="tagID" [split]="true" class="ms-1 fs-6" (click)="clickTag(tagID, $event)" (parentClick)="clickTag($event)" [clickable]="true" linkTitle="Filter by tag" i18n-title></pngx-tag>
@for (tagID of doc.tags; track tagID) {
<pngx-tag [tagID]="tagID" class="ms-1 fs-6" (click)="clickTag(tagID, $event)" [clickable]="true" linkTitle="Filter by tag" i18n-title></pngx-tag>
}
}
@case (DisplayField.DOCUMENT_TYPE) {
@@ -38,7 +38,6 @@ import {
} from 'src/app/data/filter-rule-type'
import { SavedView } from 'src/app/data/saved-view'
import { IfPermissionsDirective } from 'src/app/directives/if-permissions.directive'
import { CollapseTagsPipe } from 'src/app/pipes/collapse-tags.pipe'
import { CorrespondentNamePipe } from 'src/app/pipes/correspondent-name.pipe'
import { CustomDatePipe } from 'src/app/pipes/custom-date.pipe'
import { DocumentTitlePipe } from 'src/app/pipes/document-title.pipe'
@@ -76,7 +75,6 @@ import { WidgetFrameComponent } from '../widget-frame/widget-frame.component'
DocumentTypeNamePipe,
StoragePathNamePipe,
AsyncPipe,
CollapseTagsPipe,
DocumentTitlePipe,
CustomDatePipe,
NgClass,
@@ -390,12 +390,14 @@ describe('DocumentDetailComponent', () => {
expect(component.documentForm.get('title').value).toEqual('Foo Bar')
})
it('should update the title form value without waiting for debounce or blur', () => {
it('should update title before doc change if was not updated via debounce', () => {
jest.useFakeTimers()
initNormally()
const input = component.titleInput.inputField.nativeElement
input.value = 'Foo Bar'
input.dispatchEvent(new Event('input'))
component.titleInput.value = 'Foo Bar'
component.titleInput.inputField.nativeElement.dispatchEvent(
new Event('change')
)
jest.advanceTimersByTime(1000)
expect(component.documentForm.get('title').value).toEqual('Foo Bar')
})
@@ -718,6 +718,7 @@ describe('BulkEditorComponent', () => {
'Workflow will run on the selected documents in the background.'
)
expect(toast.mock.calls.at(-1)[0].actionName).toBeUndefined()
expect(documentListViewService.selected.size).toBe(0)
httpTestingController.match(
`${environment.apiBaseUrl}documents/?page=1&page_size=50&ordering=-created&truncate_content=true&include_selection_data=true`
) // list reload
@@ -1292,7 +1293,7 @@ describe('BulkEditorComponent', () => {
)
req.flush({ result: 'OK', task_id: 'delete-2' })
// ids aren't known when all matching documents are selected, so no undo
expect(trackSpy).toHaveBeenLastCalledWith('delete-2', 2, null)
expect(trackSpy).toHaveBeenLastCalledWith('delete-2', 0, null)
httpTestingController.match(
`${environment.apiBaseUrl}documents/?page=1&page_size=50&ordering=-created&truncate_content=true&include_selection_data=true`
) // list reload
@@ -1491,6 +1492,7 @@ describe('BulkEditorComponent', () => {
httpTestingController.match(
`${environment.apiBaseUrl}documents/?page=1&page_size=100000&fields=id`
) // listAllFilteredIds
expect(documentListViewService.selected.size).toEqual(0)
// Test with archiveFallback enabled
modal.componentInstance.deleteOriginals.set(false)
@@ -1512,6 +1514,7 @@ describe('BulkEditorComponent', () => {
httpTestingController.match(
`${environment.apiBaseUrl}documents/?page=1&page_size=100000&fields=id`
) // listAllFilteredIds
expect(documentListViewService.selected.size).toEqual(0)
})
it('should support merging documents as versions', () => {
@@ -1559,6 +1562,7 @@ describe('BulkEditorComponent', () => {
httpTestingController.match(
`${environment.apiBaseUrl}documents/?page=1&page_size=100000&fields=id`
)
expect(documentListViewService.selected.size).toEqual(0)
jest.advanceTimersByTime(400)
expect(toastSpy).toHaveBeenCalledWith(
expect.objectContaining({
@@ -1713,6 +1717,7 @@ describe('BulkEditorComponent', () => {
expect(notification.actionName).toBeUndefined()
}
expect(reload).not.toHaveBeenCalled()
expect(documentListViewService.selected.size).toBe(0)
expect(documentListViewService.allSelected).toBe(false)
TestBed.inject(TaskNotificationService).complete({
task_id: 'queued-operation',
@@ -329,6 +329,7 @@ export class BulkEditorComponent
.subscribe({
next: () => {
modal.close()
this.list.selectNone()
this.toastService.showInfo(
$localize`Workflow will run on the selected documents in the background.`
)
@@ -365,6 +366,7 @@ export class BulkEditorComponent
) {
const bulk = this.getSelectionSize() > 1
modal?.close()
this.list.selectNone()
if (onQueued) {
onQueued(taskId)
} else {
@@ -36,8 +36,8 @@
{{document().title | documentTitle}}
}
@if (displayFields().includes(DisplayField.TAGS)) {
@for (tagID of document().tags | collapseTags | async; track tagID) {
<pngx-tag [tagID]="tagID" [split]="true" linkTitle="Filter by tag" i18n-linkTitle class="ms-1" (click)="clickTag.emit(tagID);$event.stopPropagation()" (parentClick)="clickTag.emit($event)" [clickable]="clickTag.observers.length"></pngx-tag>
@for (tagID of document().tags; track tagID) {
<pngx-tag [tagID]="tagID" linkTitle="Filter by tag" i18n-linkTitle class="ms-1" (click)="clickTag.emit(tagID);$event.stopPropagation()" [clickable]="clickTag.observers.length"></pngx-tag>
}
}
} @else {
@@ -20,7 +20,6 @@ import {
Document,
} from 'src/app/data/document'
import { SETTINGS_KEYS } from 'src/app/data/ui-settings'
import { CollapseTagsPipe } from 'src/app/pipes/collapse-tags.pipe'
import { CorrespondentNamePipe } from 'src/app/pipes/correspondent-name.pipe'
import { CustomDatePipe } from 'src/app/pipes/custom-date.pipe'
import { DocumentTitlePipe } from 'src/app/pipes/document-title.pipe'
@@ -46,7 +45,6 @@ import { LoadingComponentWithPermissions } from '../../loading-component/loading
TagComponent,
CustomFieldDisplayComponent,
AsyncPipe,
CollapseTagsPipe,
NgOptimizedImage,
UsernamePipe,
CorrespondentNamePipe,
@@ -19,12 +19,11 @@
}
@if (document() && displayFields().includes(DisplayField.TAGS)) {
@let visibleTagIDs = document().tags | collapseTags | async;
<div class="tags d-flex flex-column text-end position-absolute me-1 fs-6" [class.tags-no-wrap]="visibleTagIDs?.length > 3">
@for (tagID of getTagIDs(visibleTagIDs); track tagID) {
<pngx-tag [tagID]="tagID" [split]="true" (click)="clickTag.emit(tagID);$event.stopPropagation()" (parentClick)="clickTag.emit($event)" [clickable]="true" linkTitle="Toggle tag filter" i18n-linkTitle></pngx-tag>
<div class="tags d-flex flex-column text-end position-absolute me-1 fs-6" [class.tags-no-wrap]="document().tags.length > 3">
@for (tagID of tagIDs; track tagID) {
<pngx-tag [tagID]="tagID" (click)="clickTag.emit(tagID);$event.stopPropagation()" [clickable]="true" linkTitle="Toggle tag filter" i18n-linkTitle></pngx-tag>
}
@if (getMoreTags(visibleTagIDs); as moreTags) {
@if (moreTags) {
<div>
<span class="badge text-dark">+ {{moreTags}}</span>
</div>
@@ -103,8 +103,7 @@ a {
line-height: 1;
&.tags-no-wrap {
// split-pill parents clip to a stub instead (see TagComponent)
::ng-deep .badge:not(.split-parent) {
::ng-deep .badge {
display: inline-block;
max-width: 100%;
white-space: nowrap;
@@ -4,9 +4,6 @@ import { provideHttpClientTesting } from '@angular/common/http/testing'
import { ComponentFixture, TestBed } from '@angular/core/testing'
import { By } from '@angular/platform-browser'
import { RouterTestingModule } from '@angular/router/testing'
import { of } from 'rxjs'
import { PermissionsService } from 'src/app/services/permissions.service'
import { TagService } from 'src/app/services/rest/tag.service'
import { TagComponent } from '../../common/tag/tag.component'
import { DocumentCardSmallComponent } from './document-card-small.component'
@@ -100,36 +97,8 @@ describe('DocumentCardSmallComponent', () => {
).toHaveLength(6)
})
it('should hide parent tags implied by a child and count pills for the hidden tag counter', () => {
jest
.spyOn(TestBed.inject(PermissionsService), 'currentUserCan')
.mockReturnValue(true)
jest.spyOn(TestBed.inject(TagService), 'listAll').mockReturnValue(
of({
count: 8,
all: [1, 2, 3, 4, 5, 6, 7, 8],
results: [
{ id: 1, name: 'Parent' },
{ id: 2, name: 'Child', parent: 1 },
...[3, 4, 5, 6, 7, 8].map((id) => ({ id, name: `Tag ${id}` })),
],
})
)
fixture.componentRef.setInput('document', {
...component.document(),
tags: [1, 2, 3, 4, 5, 6, 7, 8],
})
fixture.detectChanges()
const tags = fixture.debugElement.queryAll(By.directive(TagComponent))
expect(tags.map((t) => t.componentInstance.tagID())).toEqual([
2, 3, 4, 5, 6,
])
expect(fixture.nativeElement.textContent).toContain('+ 2')
})
it('should clear hidden tag counter when tag count falls below the limit', () => {
expect(component.getMoreTags(component.document().tags)).toEqual(3)
expect(component.moreTags).toEqual(3)
fixture.componentRef.setInput('document', {
...component.document(),
@@ -137,7 +106,7 @@ describe('DocumentCardSmallComponent', () => {
})
fixture.detectChanges()
expect(component.getMoreTags(component.document().tags)).toBeNull()
expect(component.moreTags).toBeNull()
expect(fixture.nativeElement.textContent).not.toContain('+ 3')
})
@@ -20,7 +20,6 @@ import {
Document,
} from 'src/app/data/document'
import { SETTINGS_KEYS } from 'src/app/data/ui-settings'
import { CollapseTagsPipe } from 'src/app/pipes/collapse-tags.pipe'
import { CorrespondentNamePipe } from 'src/app/pipes/correspondent-name.pipe'
import { CustomDatePipe } from 'src/app/pipes/custom-date.pipe'
import { DocumentTitlePipe } from 'src/app/pipes/document-title.pipe'
@@ -46,7 +45,6 @@ import { LoadingComponentWithPermissions } from '../../loading-component/loading
TagComponent,
CustomFieldDisplayComponent,
AsyncPipe,
CollapseTagsPipe,
NgOptimizedImage,
UsernamePipe,
CorrespondentNamePipe,
@@ -92,13 +90,12 @@ export class DocumentCardSmallComponent
@Output()
clickStoragePath = new EventEmitter<number>()
private get tagLimit(): number {
return this.document()?.notes.length > 0 ? 6 : 7
}
getMoreTags(tagIDs: number[]): number {
const limit = this.tagLimit
return tagIDs?.length > limit ? tagIDs.length - (limit - 1) : null
get moreTags(): number {
const document = this.document()
const limit = document?.notes.length > 0 ? 6 : 7
return document?.tags.length > limit
? document.tags.length - (limit - 1)
: null
}
@ViewChild('popupPreview') popupPreview: PreviewPopupComponent
@@ -119,9 +116,14 @@ export class DocumentCardSmallComponent
return this.documentService.getDownloadUrl(this.document().id)
}
getTagIDs(tagIDs: number[]): number[] {
const limit = this.tagLimit
return tagIDs?.length > limit ? tagIDs.slice(0, limit - 1) : tagIDs
get tagIDs() {
const document = this.document()
const limit = document.notes.length > 0 ? 6 : 7
if (document.tags.length > limit) {
return document.tags.slice(0, limit - 1)
} else {
return document.tags
}
}
mouseLeaveCard() {
@@ -335,8 +335,8 @@
<a routerLink="/documents/{{d.id}}" title="Edit document" i18n-title style="overflow-wrap: anywhere;">{{d.title | documentTitle}}</a>
}
@if (activeDisplayFields.includes(DisplayField.TAGS)) {
@for (tagID of d.tags | collapseTags | async; track tagID) {
<pngx-tag [tagID]="tagID" [split]="true" class="ms-1 fs-6" clickable="true" linkTitle="Filter by tag" i18n-linkTitle (click)="clickTag(tagID);$event.stopPropagation()" (parentClick)="clickTag($event)"></pngx-tag>
@for (tagID of d.tags; track tagID) {
<pngx-tag [tagID]="tagID" class="ms-1 fs-6" clickable="true" linkTitle="Filter by tag" i18n-linkTitle (click)="clickTag(tagID);$event.stopPropagation()"></pngx-tag>
}
}
</td>
@@ -39,7 +39,6 @@ import {
SortableDirective,
SortEvent,
} from 'src/app/directives/sortable.directive'
import { CollapseTagsPipe } from 'src/app/pipes/collapse-tags.pipe'
import { CorrespondentNamePipe } from 'src/app/pipes/correspondent-name.pipe'
import { CustomDatePipe } from 'src/app/pipes/custom-date.pipe'
import { DocumentTitlePipe } from 'src/app/pipes/document-title.pipe'
@@ -97,7 +96,6 @@ import { SaveViewConfigDialogComponent } from './save-view-config-dialog/save-vi
StoragePathNamePipe,
BootstrapIconComponent,
AsyncPipe,
CollapseTagsPipe,
FormsModule,
ReactiveFormsModule,
NgTemplateOutlet,
-6
View File
@@ -94,7 +94,6 @@ export const SETTINGS_KEYS = {
DEFAULT_PERMS_VIEW_GROUPS: 'general-settings:permissions:default-view-groups',
DEFAULT_PERMS_EDIT_USERS: 'general-settings:permissions:default-edit-users',
DEFAULT_PERMS_EDIT_GROUPS: 'general-settings:permissions:default-edit-groups',
TAGS_SHOW_HIERARCHY: 'general-settings:tags:show-hierarchy',
DOCUMENT_EDITING_REMOVE_INBOX_TAGS:
'general-settings:document-editing:remove-inbox-tags',
DOCUMENT_EDITING_OVERLAY_THUMBNAIL:
@@ -306,11 +305,6 @@ export const SETTINGS: UiSetting[] = [
type: 'string',
default: '',
},
{
key: SETTINGS_KEYS.TAGS_SHOW_HIERARCHY,
type: 'boolean',
default: true,
},
{
key: SETTINGS_KEYS.DOCUMENT_EDITING_REMOVE_INBOX_TAGS,
type: 'boolean',
@@ -1,67 +0,0 @@
import { provideHttpClient, withInterceptorsFromDi } from '@angular/common/http'
import { provideHttpClientTesting } from '@angular/common/http/testing'
import { TestBed } from '@angular/core/testing'
import { firstValueFrom, of, throwError } from 'rxjs'
import { Tag } from '../data/tag'
import { PermissionsService } from '../services/permissions.service'
import { TagService } from '../services/rest/tag.service'
import { SettingsService } from '../services/settings.service'
import { CollapseTagsPipe } from './collapse-tags.pipe'
const tags: Tag[] = [
{ id: 1, name: 'Parent' },
{ id: 2, name: 'Child', parent: 1 },
{ id: 3, name: 'Other' },
]
describe('CollapseTagsPipe', () => {
let pipe: CollapseTagsPipe
let tagService: TagService
let settingsService: SettingsService
let permissionsService: PermissionsService
beforeEach(() => {
TestBed.configureTestingModule({
providers: [
CollapseTagsPipe,
provideHttpClient(withInterceptorsFromDi()),
provideHttpClientTesting(),
],
})
pipe = TestBed.inject(CollapseTagsPipe)
tagService = TestBed.inject(TagService)
settingsService = TestBed.inject(SettingsService)
permissionsService = TestBed.inject(PermissionsService)
jest.spyOn(permissionsService, 'currentUserCan').mockReturnValue(true)
jest.spyOn(settingsService, 'get').mockReturnValue(true)
jest
.spyOn(tagService, 'listAll')
.mockReturnValue(of({ count: 3, all: [1, 2, 3], results: tags }))
})
it('should drop parents implied by a selected child', async () => {
expect(await firstValueFrom(pipe.transform([1, 2, 3]))).toEqual([2, 3])
})
it('should pass ids through when hierarchy display is off or not permitted', async () => {
jest.spyOn(settingsService, 'get').mockReturnValue(false)
expect(await firstValueFrom(pipe.transform([1, 2]))).toEqual([1, 2])
jest.spyOn(settingsService, 'get').mockReturnValue(true)
jest.spyOn(permissionsService, 'currentUserCan').mockReturnValue(false)
expect(await firstValueFrom(pipe.transform([1, 2]))).toEqual([1, 2])
expect(tagService.listAll).not.toHaveBeenCalled()
})
it('should handle empty input', async () => {
expect(await firstValueFrom(pipe.transform(null))).toEqual([])
expect(await firstValueFrom(pipe.transform([]))).toEqual([])
})
it('should fall back to the original ids if tags fail to load', async () => {
jest
.spyOn(tagService, 'listAll')
.mockReturnValue(throwError(() => new Error('failed')))
expect(await firstValueFrom(pipe.transform([1, 2]))).toEqual([1, 2])
})
})
@@ -1,47 +0,0 @@
import { inject, Pipe, PipeTransform } from '@angular/core'
import { catchError, map, Observable, of } from 'rxjs'
import { Tag } from '../data/tag'
import { SETTINGS_KEYS } from '../data/ui-settings'
import {
PermissionAction,
PermissionsService,
PermissionType,
} from '../services/permissions.service'
import { TagService } from '../services/rest/tag.service'
import { SettingsService } from '../services/settings.service'
import { collapseImpliedAncestors } from '../utils/collapse-tags'
/**
* Drops tag IDs whose descendant is also in the list for when the child's
* Tag badge already shows its parent.
*/
@Pipe({
name: 'collapseTags',
})
export class CollapseTagsPipe implements PipeTransform {
private readonly permissionsService = inject(PermissionsService)
private readonly tagService = inject(TagService)
private readonly settingsService = inject(SettingsService)
transform(ids: number[]): Observable<number[]> {
if (
!ids?.length ||
!this.settingsService.get(SETTINGS_KEYS.TAGS_SHOW_HIERARCHY) ||
!this.permissionsService.currentUserCan(
PermissionAction.View,
PermissionType.Tag
)
) {
return of(ids ?? [])
}
return this.tagService.listAll().pipe(
map((tags) =>
collapseImpliedAncestors(
ids,
new Map<number, Tag>(tags.results.map((t) => [t.id, t]))
)
),
catchError(() => of(ids))
)
}
}
@@ -1,47 +0,0 @@
import { Tag } from '../data/tag'
import { collapseImpliedAncestors, getTagAncestors } from './collapse-tags'
const tags: Tag[] = [
{ id: 1, name: 'Finance' },
{ id: 2, name: 'Account', parent: 1 },
{ id: 3, name: 'Mortgage', parent: 2 },
{ id: 4, name: 'Taxes', parent: 1 },
{ id: 5, name: 'Inbox' },
]
const tagsById = new Map(tags.map((t) => [t.id, t]))
describe('collapseImpliedAncestors', () => {
it('drops ancestors that are implied by a descendant in the list', () => {
expect(collapseImpliedAncestors([1, 2, 3, 4, 5], tagsById)).toEqual([
3, 4, 5,
])
})
it('keeps a parent with none of its children present', () => {
expect(collapseImpliedAncestors([1, 5], tagsById)).toEqual([1, 5])
})
it('keeps unknown (private) ids and handles empty input', () => {
expect(collapseImpliedAncestors([99, 2], tagsById)).toEqual([99, 2])
expect(collapseImpliedAncestors([], tagsById)).toEqual([])
expect(collapseImpliedAncestors(null, tagsById)).toEqual([])
})
})
describe('getTagAncestors', () => {
it('returns ancestors root first', () => {
expect(getTagAncestors(3, tagsById).map((t) => t.id)).toEqual([1, 2])
expect(getTagAncestors(5, tagsById)).toEqual([])
})
it('stops at a parent that is not visible or forms a cycle', () => {
const hidden = new Map<number, Tag>([[6, { id: 6, name: 'x', parent: 99 }]])
expect(getTagAncestors(6, hidden)).toEqual([])
const cyclic = new Map<number, Tag>([
[7, { id: 7, name: 'a', parent: 8 }],
[8, { id: 8, name: 'b', parent: 7 }],
])
expect(getTagAncestors(7, cyclic).map((t) => t.id)).toEqual([7, 8])
})
})
-43
View File
@@ -1,43 +0,0 @@
import { Tag } from '../data/tag'
/**
* Collapses implied ancestor tags from a list of tag IDs such that if a tag's
* ancestor is also present in the list, the ancestor is removed. For example
* [Finance, Account, Mortgage] -> [Mortgage]
*/
export function collapseImpliedAncestors(
ids: number[],
tagsById: Map<number, Tag>
): number[] {
if (!ids?.length) return ids ?? []
const present = new Set(ids)
const implied = new Set<number>()
for (const id of ids) {
const seen = new Set<number>()
let parentId = tagsById.get(id)?.parent
while (parentId && !seen.has(parentId)) {
seen.add(parentId)
if (present.has(parentId)) implied.add(parentId)
parentId = tagsById.get(parentId)?.parent
}
}
return ids.filter((id) => !implied.has(id))
}
/**
* Returns an array of ancestor tags for the given tag ID, starting from the
* root and ending with the immediate parent.
*/
export function getTagAncestors(id: number, tagsById: Map<number, Tag>): Tag[] {
const chain: Tag[] = []
const seen = new Set<number>()
let parentId = tagsById.get(id)?.parent
while (parentId && !seen.has(parentId)) {
seen.add(parentId)
const parent = tagsById.get(parentId)
if (!parent) break
chain.unshift(parent)
parentId = parent.parent
}
return chain
}
+1 -1
View File
@@ -8,7 +8,7 @@ export const environment = {
apiVersion: '10', // match src/paperless/settings.py
appTitle: DEFAULT_APP_TITLE,
tag: 'prod',
version: '3.5.1',
version: '3.4.2',
webSocketHost: window.location.host,
webSocketProtocol: window.location.protocol == 'https:' ? 'wss:' : 'ws:',
webSocketBaseUrl: base_url.pathname + 'ws/',
File diff suppressed because it is too large. Load diff
File diff suppressed because it is too large. Load diff
File diff suppressed because it is too large. Load diff
File diff suppressed because it is too large. Load diff
File diff suppressed because it is too large. Load diff
File diff suppressed because it is too large. Load diff
File diff suppressed because it is too large. Load diff
File diff suppressed because it is too large. Load diff
File diff suppressed because it is too large. Load diff
File diff suppressed because it is too large. Load diff
File diff suppressed because it is too large. Load diff
File diff suppressed because it is too large. Load diff
File diff suppressed because it is too large. Load diff
File diff suppressed because it is too large. Load diff
File diff suppressed because it is too large. Load diff
File diff suppressed because it is too large. Load diff
File diff suppressed because it is too large. Load diff
File diff suppressed because it is too large. Load diff
File diff suppressed because it is too large. Load diff
File diff suppressed because it is too large. Load diff
File diff suppressed because it is too large. Load diff
File diff suppressed because it is too large. Load diff
File diff suppressed because it is too large. Load diff
File diff suppressed because it is too large. Load diff
File diff suppressed because it is too large. Load diff
File diff suppressed because it is too large. Load diff
File diff suppressed because it is too large. Load diff
File diff suppressed because it is too large. Load diff
File diff suppressed because it is too large. Load diff
File diff suppressed because it is too large. Load diff
File diff suppressed because it is too large. Load diff
File diff suppressed because it is too large. Load diff
File diff suppressed because it is too large. Load diff
File diff suppressed because it is too large. Load diff
File diff suppressed because it is too large. Load diff
File diff suppressed because it is too large. Load diff
File diff suppressed because it is too large. Load diff
File diff suppressed because it is too large. Load diff
File diff suppressed because it is too large. Load diff
File diff suppressed because it is too large. Load diff
File diff suppressed because it is too large. Load diff
File diff suppressed because it is too large. Load diff
File diff suppressed because it is too large. Load diff
File diff suppressed because it is too large. Load diff
File diff suppressed because it is too large. Load diff
File diff suppressed because it is too large. Load diff
File diff suppressed because it is too large. Load diff
+126 -69
View File
@@ -39,6 +39,8 @@ from documents.models import Tag
from documents.permissions import PermissionsSpec
from documents.permissions import set_permissions_for_objects
from documents.plugins.helpers import DocumentsStatusManager
from documents.task_tracking import queue_consumption
from documents.task_tracking import queue_processing_tasks
from documents.tasks import bulk_update_documents
from documents.tasks import consume_file
from documents.tasks import remove_document_from_index
@@ -432,7 +434,12 @@ def delete(doc_ids: list[int]) -> Literal["OK"]:
return "OK"
def reprocess(doc_ids: list[int], *, remote_ocr: bool = False) -> Literal["OK"]:
def reprocess(
doc_ids: list[int],
*,
remote_ocr: bool = False,
parent_task_id: str | None = None,
) -> Literal["OK"]:
"""
Re-run parsing for the given documents.
@@ -457,6 +464,16 @@ def reprocess(doc_ids: list[int], *, remote_ocr: bool = False) -> Literal["OK"]:
.values_list("source_id", flat=True)
.distinct()
)
if parent_task_id is not None:
from documents.task_tracking import queue_reprocessing_tasks
queue_reprocessing_tasks(
parent_task_id,
list(source_ids),
remote_ocr=remote_ocr,
)
return "OK"
for source_id in source_ids:
update_document_content_maybe_archive_file.apply_async(
kwargs={"document_id": source_id, "remote_ocr": remote_ocr},
@@ -521,6 +538,7 @@ def rotate(
source_mode: SourceMode = SourceModeChoices.LATEST_VERSION,
user: User | None = None,
trigger_source: PaperlessTask.TriggerSource = PaperlessTask.TriggerSource.WEB_UI,
parent_task_id: str | None = None,
) -> Literal["OK"]:
logger.info(
f"Attempting to rotate {len(doc_ids)} documents by {degrees} degrees.",
@@ -539,7 +557,6 @@ def rotate(
pair = _resolve_root_and_source_doc(doc, source_mode=source_mode)
docs_by_root_id.setdefault(pair.root_doc.id, pair)
failed_ids = []
for pair in docs_by_root_id.values():
if pair.source_doc.mime_type != "application/pdf":
logger.warning(
@@ -559,28 +576,22 @@ def rotate(
if user is not None:
overrides.actor_id = user.id
consume_file.apply_async(
kwargs={
"input_doc": ConsumableDocument(
source=DocumentSource.ConsumeFolder,
original_file=filepath,
root_document_id=pair.root_doc.id,
),
"overrides": overrides,
},
headers={"trigger_source": trigger_source},
queue_consumption(
input_doc=ConsumableDocument(
source=DocumentSource.ConsumeFolder,
original_file=filepath,
root_document_id=pair.root_doc.id,
),
overrides=overrides,
trigger_source=trigger_source,
parent_task_id=parent_task_id,
)
logger.info(
f"Queued new rotated version for document {pair.root_doc.id} by {degrees} degrees",
)
except Exception as e:
failed_ids.append(pair.root_doc.id)
logger.exception(f"Error rotating document {pair.root_doc.id}: {e}")
if failed_ids:
raise RuntimeError(
f"Rotation failed for document(s): {failed_ids}. Check logs for details.",
)
raise
return "OK"
@@ -594,13 +605,15 @@ def merge(
source_mode: SourceMode = SourceModeChoices.LATEST_VERSION,
user: User | None = None,
trigger_source: PaperlessTask.TriggerSource = PaperlessTask.TriggerSource.WEB_UI,
) -> Literal["OK"]:
parent_task_id: str | None = None,
) -> Literal["OK"] | dict:
logger.info(
f"Attempting to merge {len(doc_ids)} documents into a single document.",
)
qs = Document.objects.select_related("root_document").filter(id__in=doc_ids)
docs_by_id = {doc.id: doc for doc in qs}
affected_docs: list[int] = []
failed_ids: list[int] = []
handoff_asn: int | None = None
with pdf_ops.PdfMerger() as merger:
# use doc_ids to preserve order
@@ -626,11 +639,14 @@ def merge(
if handoff_asn is None and doc.archive_serial_number is not None:
handoff_asn = doc.archive_serial_number
except Exception as e:
failed_ids.append(doc.id)
logger.exception(
f"Error merging document {doc.id}, it will not be included in the merge: {e}",
)
if len(affected_docs) == 0:
logger.warning("No documents were merged")
if parent_task_id is not None:
raise ValueError("None of the selected documents could be merged.")
return "OK"
filepath = (
@@ -673,6 +689,23 @@ def merge(
overrides=overrides,
).set(headers={"trigger_source": trigger_source})
if parent_task_id is not None:
backup = (
release_archive_serial_numbers(affected_docs) if delete_originals else {}
)
queue_processing_tasks(
parent_task_id,
[consume_task],
delete_ids=affected_docs if delete_originals else None,
asn_backup=backup,
)
if failed_ids:
return {
"failed_document_ids": failed_ids,
"error_message": f"Documents {failed_ids} were excluded from the merge.",
}
return "OK"
if delete_originals:
backup = release_archive_serial_numbers(affected_docs)
logger.info(
@@ -692,22 +725,6 @@ def merge(
return "OK"
def validate_merge_as_versions_state(
documents: list[Document],
root_document_id: int,
) -> None:
if any(document.root_document_id is not None for document in documents):
raise ValueError("Only top-level documents can be merged as versions.")
source_ids = [
document.pk for document in documents if document.pk != root_document_id
]
if Document.global_objects.filter(root_document_id__in=source_ids).exists():
raise ValueError(
"Documents with existing versions cannot be merged into another document.",
)
def merge_as_versions(
doc_ids: list[int],
*,
@@ -725,8 +742,15 @@ def merge_as_versions(
)
documents_by_id = {document.id: document for document in documents}
validate_merge_as_versions_state(documents, root_document_id)
if any(document.root_document_id is not None for document in documents):
raise ValueError("Only top-level documents can be merged as versions.")
source_ids = [doc_id for doc_id in doc_ids if doc_id != root_document_id]
if Document.global_objects.filter(root_document_id__in=source_ids).exists():
raise ValueError(
"Documents with existing versions cannot be merged into another document.",
)
root_document = documents_by_id[root_document_id]
next_version_index = (
Document.global_objects.filter(
@@ -826,6 +850,7 @@ def split(
source_mode: SourceMode = SourceModeChoices.LATEST_VERSION,
user: User | None = None,
trigger_source: PaperlessTask.TriggerSource = PaperlessTask.TriggerSource.WEB_UI,
parent_task_id: str | None = None,
) -> Literal["OK"]:
logger.info(
f"Attempting to split document {doc_ids[0]} into {len(pages)} documents",
@@ -869,7 +894,17 @@ def split(
).set(headers={"trigger_source": trigger_source}),
)
if delete_originals:
if parent_task_id is not None:
backup = (
release_archive_serial_numbers([doc.id]) if delete_originals else {}
)
queue_processing_tasks(
parent_task_id,
consume_tasks,
delete_ids=[doc.id] if delete_originals else None,
asn_backup=backup,
)
elif delete_originals:
backup = release_archive_serial_numbers([doc.id])
logger.info(
"Queueing removal of original document after consumption of the split documents",
@@ -901,6 +936,7 @@ def delete_pages(
source_mode: SourceMode = SourceModeChoices.LATEST_VERSION,
user: User | None = None,
trigger_source: PaperlessTask.TriggerSource = PaperlessTask.TriggerSource.WEB_UI,
parent_task_id: str | None = None,
) -> Literal["OK"]:
logger.info(
f"Attempting to delete pages {pages} from {len(doc_ids)} documents",
@@ -919,16 +955,15 @@ def delete_pages(
overrides = DocumentMetadataOverrides().from_document(pair.root_doc)
if user is not None:
overrides.actor_id = user.id
consume_file.apply_async(
kwargs={
"input_doc": ConsumableDocument(
source=DocumentSource.ConsumeFolder,
original_file=filepath,
root_document_id=pair.root_doc.id,
),
"overrides": overrides,
},
headers={"trigger_source": trigger_source},
queue_consumption(
input_doc=ConsumableDocument(
source=DocumentSource.ConsumeFolder,
original_file=filepath,
root_document_id=pair.root_doc.id,
),
overrides=overrides,
trigger_source=trigger_source,
parent_task_id=parent_task_id,
)
logger.info(
f"Queued new version for document {pair.root_doc.id} after deleting pages {pages}",
@@ -950,6 +985,7 @@ def edit_pdf(
source_mode: SourceMode = SourceModeChoices.LATEST_VERSION,
user: User | None = None,
trigger_source: PaperlessTask.TriggerSource = PaperlessTask.TriggerSource.WEB_UI,
parent_task_id: str | None = None,
) -> Literal["OK"]:
"""
Operations is a list of dictionaries describing the final PDF pages.
@@ -994,16 +1030,15 @@ def edit_pdf(
if user is not None:
overrides.owner_id = user.id
overrides.actor_id = user.id
consume_file.apply_async(
kwargs={
"input_doc": ConsumableDocument(
source=DocumentSource.ConsumeFolder,
original_file=filepath,
root_document_id=pair.root_doc.id,
),
"overrides": overrides,
},
headers={"trigger_source": trigger_source},
queue_consumption(
input_doc=ConsumableDocument(
source=DocumentSource.ConsumeFolder,
original_file=filepath,
root_document_id=pair.root_doc.id,
),
overrides=overrides,
trigger_source=trigger_source,
parent_task_id=parent_task_id,
)
else:
version_filepaths = pdf_ops.build_pdfs(
@@ -1043,7 +1078,17 @@ def edit_pdf(
).set(headers={"trigger_source": trigger_source}),
)
if delete_original:
if parent_task_id is not None:
backup = (
release_archive_serial_numbers([doc.id]) if delete_original else {}
)
queue_processing_tasks(
parent_task_id,
consume_tasks,
delete_ids=[doc.id] if delete_original else None,
asn_backup=backup,
)
elif delete_original:
backup = release_archive_serial_numbers([doc.id])
try:
chord(
@@ -1077,6 +1122,7 @@ def remove_password(
source_mode: SourceMode = SourceModeChoices.LATEST_VERSION,
user: User | None = None,
trigger_source: PaperlessTask.TriggerSource = PaperlessTask.TriggerSource.WEB_UI,
parent_task_id: str | None = None,
source_paths_by_id: Mapping[int, Path] | None = None,
) -> Literal["OK"]:
"""
@@ -1119,16 +1165,15 @@ def remove_password(
if user is not None:
overrides.owner_id = user.id
overrides.actor_id = user.id
consume_file.apply_async(
kwargs={
"input_doc": ConsumableDocument(
source=DocumentSource.ConsumeFolder,
original_file=filepath,
root_document_id=pair.root_doc.id,
),
"overrides": overrides,
},
headers={"trigger_source": trigger_source},
queue_consumption(
input_doc=ConsumableDocument(
source=DocumentSource.ConsumeFolder,
original_file=filepath,
root_document_id=pair.root_doc.id,
),
overrides=overrides,
trigger_source=trigger_source,
parent_task_id=parent_task_id,
)
else:
consume_tasks = []
@@ -1151,7 +1196,19 @@ def remove_password(
).set(headers={"trigger_source": trigger_source}),
)
if delete_original:
if parent_task_id is not None:
backup = (
release_archive_serial_numbers([doc.id])
if delete_original
else {}
)
queue_processing_tasks(
parent_task_id,
consume_tasks,
delete_ids=[doc.id] if delete_original else None,
asn_backup=backup,
)
elif delete_original:
chord(
header=consume_tasks,
body=delete.si([doc.id]),
+15 -8
View File
@@ -1,6 +1,5 @@
"""Queued document bulk operations."""
from contextlib import nullcontext
from typing import Any
from auditlog.context import set_actor
@@ -90,8 +89,9 @@ def snapshot_field(doc_ids: list[int], field: str) -> dict[int, Any]:
return values
@shared_task
@shared_task(bind=True)
def execute_bulk_operation(
self,
operation: str,
document_ids: list[int],
parameters: dict,
@@ -113,22 +113,25 @@ def execute_bulk_operation(
if operation == "run_workflow" and not user.has_perm("documents.view_workflow"):
raise PermissionDenied("Insufficient permissions to run this workflow")
actor_context = set_actor(user) if settings.AUDIT_LOG_ENABLED else nullcontext()
if operation not in MODIFIED_FIELDS:
execution_parameters = parameters.copy()
if (
operation in TRIGGER_SOURCE_OPERATIONS | {"reprocess"}
and self.request.id is not None
):
execution_parameters["parent_task_id"] = self.request.id
if operation in USER_OPERATIONS:
execution_parameters["user"] = user
if operation in TRIGGER_SOURCE_OPERATIONS:
execution_parameters["trigger_source"] = trigger_source
modified_field = LEGACY_MODIFIED_FIELDS.get(operation)
with actor_context:
with set_actor(user):
old_values = (
snapshot_field(document_ids, modified_field)
if settings.AUDIT_LOG_ENABLED and modified_field
else {}
)
method(document_ids, **execution_parameters)
result = method(document_ids, **execution_parameters)
if settings.AUDIT_LOG_ENABLED and modified_field:
from auditlog.models import LogEntry
@@ -143,13 +146,17 @@ def execute_bulk_operation(
actor=user,
additional_data={"reason": f"Bulk edit: {operation}"},
)
return {"operation": operation, "document_count": len(document_ids)}
return {
"operation": operation,
"document_count": len(document_ids),
**(result if isinstance(result, dict) else {}),
}
affected_ids: list[int] = []
modified_field = MODIFIED_FIELDS[operation]
# Keep metadata writes and their audit entries together. Follow-up processing
# runs after commit, within this same tracked task.
with actor_context, transaction.atomic():
with set_actor(user), transaction.atomic():
old_values = (
snapshot_field(document_ids, modified_field)
if settings.AUDIT_LOG_ENABLED and modified_field
+3 -45
View File
@@ -3,6 +3,7 @@ from __future__ import annotations
import logging
import re
from fnmatch import fnmatch
from fnmatch import translate as fnmatch_translate
from typing import TYPE_CHECKING
from rest_framework import serializers
@@ -547,48 +548,6 @@ def existing_document_matches_workflow(
return (True, None)
def _glob_to_portable_regex(pattern: str) -> str:
"""
Translate an fnmatch-style glob into an anchored regex which is accepted by
Python, PostgreSQL (ARE), and MariaDB (PCRE).
fnmatch.translate() can't be used, as its output (`(?s:...)\\Z`) uses syntax
PostgreSQL rejects. Supports `*`, `?`, `[seq]` and `[!seq]`.
"""
out: list[str] = []
i = 0
n = len(pattern)
while i < n:
c = pattern[i]
i += 1
if c == "*":
out.append(".*")
elif c == "?":
out.append(".")
elif c == "[":
j = i
if j < n and pattern[j] == "!":
j += 1
if j < n and pattern[j] == "]":
j += 1
while j < n and pattern[j] != "]":
j += 1
if j >= n:
# No closing bracket, treat as a literal
out.append(r"\[")
else:
body = pattern[i:j]
i = j + 1
negate = body.startswith("!")
if negate:
body = body[1:]
body = "".join(f"\\{ch}" if ch in "\\[^" else ch for ch in body)
out.append(f"[{'^' if negate else ''}{body}]")
else:
out.append(re.escape(c))
return f"^{''.join(out)}$"
def prefilter_documents_by_workflowtrigger(
documents: QuerySet[Document],
trigger: WorkflowTrigger,
@@ -669,9 +628,8 @@ def prefilter_documents_by_workflowtrigger(
documents = documents.annotate(**annotations).filter(custom_field_q)
if trigger.filter_filename:
documents = documents.filter(
original_filename__iregex=_glob_to_portable_regex(trigger.filter_filename),
)
regex = fnmatch_translate(trigger.filter_filename).lstrip("^").rstrip("$")
documents = documents.filter(original_filename__iregex=regex)
return documents
@@ -0,0 +1,30 @@
# Generated by Django 5.2.18 on 2026-11-14 19:59
import django.db.models.deletion
from django.db import migrations
from django.db import models
class Migration(migrations.Migration):
dependencies = [
("documents", "0030_note_format"),
]
operations = [
migrations.AddField(
model_name="paperlesstask",
name="dispatched",
field=models.BooleanField(default=False),
),
migrations.AddField(
model_name="paperlesstask",
name="parent",
field=models.ForeignKey(
blank=True,
null=True,
on_delete=django.db.models.deletion.SET_NULL,
related_name="children",
to="documents.paperlesstask",
),
),
]
+10
View File
@@ -970,6 +970,16 @@ class PaperlessTask(ModelWithOwner):
help_text=_("Structured result data from task execution"),
)
parent = models.ForeignKey(
"self",
null=True,
blank=True,
on_delete=models.SET_NULL,
related_name="children",
)
dispatched = models.BooleanField(default=False)
# Acknowledgment
acknowledged = models.BooleanField(
default=False,
+2 -11
View File
@@ -1717,17 +1717,8 @@ class MergeDocumentsAsVersionsSerializer(DocumentListSerializer):
"root_document_id must be one of the selected documents.",
)
# Rechecked under row locks during execution in case document state changes.
selected_documents = list(
Document.objects.filter(id__in=documents).only("id", "root_document"),
)
try:
bulk_edit.validate_merge_as_versions_state(
selected_documents,
attrs["root_document_id"],
)
except ValueError as error:
raise serializers.ValidationError(str(error)) from error
# Version state checks (e.g., whether a document is already part of a version chain)
# moved to merge_as_versions() under row locks, since it could change between queueing and execution
return attrs
+44
View File
@@ -56,6 +56,8 @@ from documents.models import WorkflowRun
from documents.models import WorkflowTrigger
from documents.permissions import restrict_queryset_to_visible
from documents.plugins.helpers import DocumentsStatusManager
from documents.task_tracking import finish_dispatch
from documents.task_tracking import update_parent
from documents.templating.utils import convert_format_str_to_template_format
from documents.utils import compute_checksum
from documents.utils import copy_file_with_basic_stats
@@ -1077,6 +1079,7 @@ TRACKED_TASKS: dict[str, PaperlessTask.TaskType] = {
"documents.tasks.update_document_content_maybe_archive_file": PaperlessTask.TaskType.REPROCESS_DOCUMENT,
"documents.tasks.build_share_link_bundle": PaperlessTask.TaskType.BUILD_SHARE_LINK,
"documents.bulk_edit.delete": PaperlessTask.TaskType.BULK_DELETE,
"documents.task_tracking.finish_file_operation": PaperlessTask.TaskType.BULK_DELETE,
"documents.tasks.apply_ai_suggestions": PaperlessTask.TaskType.APPLY_AI_SUGGESTIONS,
BULK_OPERATION_TASK_NAME: PaperlessTask.TaskType.BULK_UPDATE,
}
@@ -1125,6 +1128,8 @@ def _extract_input_data(
data["overrides"] = override_dict
return data
if task_type == PaperlessTask.TaskType.REPROCESS_DOCUMENT:
return {"document_id": task_kwargs["document_id"]}
if task_type == PaperlessTask.TaskType.MAIL_FETCH:
account_ids = task_kwargs.get("account_ids")
if account_ids is not None:
@@ -1212,6 +1217,19 @@ def before_task_publish_handler(
input_data = _extract_input_data(task_type, task_kwargs)
owner_id = _extract_owner_id(task_type, task_kwargs)
trigger_source = _determine_trigger_source(headers)
# Consumption can fan out again (e.g. barcode splitting). Keep those
# descendants attached so their caller cannot finish before them.
parent = None
if task_type == PaperlessTask.TaskType.CONSUME_FILE and headers.get(
"parent_id",
):
parent = PaperlessTask.objects.filter(
task_id=headers["parent_id"],
parent__isnull=False,
).first()
if parent is not None:
owner_id = parent.owner_id
trigger_source = parent.trigger_source
# A retried task is republished with the same task_id, so this fires
# again for it; get_or_create keeps the original PENDING record
@@ -1224,6 +1242,7 @@ def before_task_publish_handler(
"status": PaperlessTask.Status.PENDING,
"input_data": input_data,
"owner_id": owner_id,
"parent": parent,
},
)
except Exception: # pragma: no cover
@@ -1309,6 +1328,10 @@ def task_postrun_handler(
except PaperlessTask.DoesNotExist:
return
if task_instance.children.exists():
finish_dispatch(task_instance, retval if isinstance(retval, dict) else {})
return
task_instance.status = new_status
task_instance.date_done = now
changed_fields = ["status", "date_done"]
@@ -1331,7 +1354,21 @@ def task_postrun_handler(
task_instance.status = PaperlessTask.Status.FAILURE
changed_fields.append("status")
if (
task_instance.parent_id is not None
and task_instance.task_type == PaperlessTask.TaskType.CONSUME_FILE
and (not isinstance(retval, dict) or "document_id" not in retval)
and not (isinstance(retval, dict) and "duplicate_of" in retval)
):
task_instance.status = PaperlessTask.Status.FAILURE
task_instance.result_data = {
**(retval if isinstance(retval, dict) else {}),
"error_message": "Consumption stopped without creating a document.",
}
changed_fields.extend(["status", "result_data"])
task_instance.save(update_fields=changed_fields)
update_parent(task_instance)
if task and task.name == BULK_OPERATION_TASK_NAME:
_send_bulk_operation_status(task_instance)
except Exception: # pragma: no cover
@@ -1388,6 +1425,10 @@ def task_failure_handler(
}
task_qs = PaperlessTask.objects.filter(task_id=task_id)
parent = task_qs.first()
if parent is not None and parent.children.exists():
finish_dispatch(parent, result_data)
return
task_instance = task_qs.values("date_started", "date_created").first()
if task_instance:
date_started = task_instance["date_started"]
@@ -1399,6 +1440,7 @@ def task_failure_handler(
date_started - date_created
).total_seconds()
task_qs.update(**update_fields)
update_parent(task_qs.get())
if sender and sender.name == BULK_OPERATION_TASK_NAME:
_send_bulk_operation_status(task_qs.first())
except Exception: # pragma: no cover
@@ -1445,6 +1487,8 @@ def task_revoked_handler(
"reason": (reason + " Check worker logs for details."),
},
)
if task_instance := task_qs.first():
update_parent(task_instance)
if sender and sender.name == BULK_OPERATION_TASK_NAME:
_send_bulk_operation_status(task_qs.first())
except Exception: # pragma: no cover
+242
View File
@@ -0,0 +1,242 @@
"""Completion tracking for operations which dispatch document processing tasks."""
from __future__ import annotations
import uuid
from typing import TYPE_CHECKING
from celery import shared_task
from django.db import transaction
from django.utils import timezone
from documents.models import PaperlessTask
if TYPE_CHECKING:
from celery.canvas import Signature
from documents.data_models import ConsumableDocument
from documents.data_models import DocumentMetadataOverrides
def queue_consumption(
*,
input_doc: ConsumableDocument,
overrides: DocumentMetadataOverrides,
trigger_source: str,
parent_task_id: str | None,
) -> None:
from documents.tasks import consume_file
kwargs = {"input_doc": input_doc, "overrides": overrides}
if parent_task_id is None:
consume_file.apply_async(
kwargs=kwargs,
headers={"trigger_source": trigger_source},
)
else:
queue_processing_tasks(parent_task_id, [consume_file.s(**kwargs)])
def queue_reprocessing_tasks(
parent_task_id: str,
document_ids: list[int],
*,
remote_ocr: bool,
) -> None:
from documents.tasks import update_document_content_maybe_archive_file
queue_processing_tasks(
parent_task_id,
[
update_document_content_maybe_archive_file.s(
document_id=document_id,
remote_ocr=remote_ocr,
)
for document_id in document_ids
],
)
def queue_processing_tasks(
parent_task_id: str,
signatures: list[Signature],
*,
delete_ids: list[int] | None = None,
asn_backup: dict | None = None,
) -> None:
from documents.signals.handlers import TRACKED_TASKS
from documents.signals.handlers import _extract_input_data
parent = PaperlessTask.objects.get(task_id=parent_task_id)
children = [
PaperlessTask(
task_id=str(uuid.uuid4()),
task_type=TRACKED_TASKS[signature.task],
trigger_source=parent.trigger_source,
owner_id=parent.owner_id,
parent=parent,
input_data=_extract_input_data(
TRACKED_TASKS[signature.task],
signature.kwargs,
),
)
for signature in signatures
]
# Dependencies and their final step must be registered together.
with transaction.atomic():
PaperlessTask.objects.bulk_create(children)
if delete_ids:
PaperlessTask.objects.create(
task_id=str(uuid.uuid4()),
task_type=PaperlessTask.TaskType.BULK_DELETE,
owner_id=parent.owner_id,
trigger_source=parent.trigger_source,
parent=parent,
input_data={
"dependencies": [child.pk for child in children],
"delete_ids": delete_ids,
"asn_backup": asn_backup or {},
},
)
for index, (signature, child) in enumerate(zip(signatures, children, strict=True)):
try:
signature.apply_async(
task_id=child.task_id,
headers={"trigger_source": parent.trigger_source},
)
except Exception:
PaperlessTask.objects.filter(
pk__in=[task.pk for task in children[index:]],
).update(
status=PaperlessTask.Status.FAILURE,
date_done=timezone.now(),
result_data={"error_message": "Could not publish processing task."},
)
raise
@shared_task
def finish_file_operation(
dependencies: list[int],
delete_ids: list[int],
asn_backup: dict,
) -> dict[str, str]:
from documents.bulk_edit import delete
from documents.bulk_edit import restore_archive_serial_numbers
statuses = list(
PaperlessTask.objects.filter(pk__in=dependencies).values_list(
"status",
flat=True,
),
)
if len(statuses) != len(dependencies):
raise ValueError(
"Processing task records are missing; originals were preserved.",
)
if any(status != PaperlessTask.Status.SUCCESS for status in statuses):
restore_archive_serial_numbers(asn_backup)
return {"reason": "Originals preserved; archive serial numbers restored."}
delete(delete_ids)
return {"reason": "Originals moved to trash."}
def _publish_finalizer(task: PaperlessTask) -> None:
try:
finish_file_operation.apply_async(
task_id=task.task_id,
kwargs=task.input_data,
headers={"trigger_source": task.trigger_source},
)
except Exception as error:
from documents.signals.handlers import task_failure_handler
task_failure_handler(
sender=finish_file_operation,
task_id=task.task_id,
exception=error,
)
def _schedule_finalizers(parent: PaperlessTask) -> None:
for task in parent.children.filter(
task_type=PaperlessTask.TaskType.BULK_DELETE,
status=PaperlessTask.Status.PENDING,
):
dependencies = task.input_data.get("dependencies")
if dependencies is None or task.result_data:
continue
if (
PaperlessTask.objects.filter(pk__in=dependencies)
.exclude(
status__in=PaperlessTask.COMPLETE_STATUSES,
)
.exists()
):
continue
task.result_data = {"scheduled": True}
task.save(update_fields=["result_data"])
transaction.on_commit(lambda task=task: _publish_finalizer(task))
def finish_dispatch(task: PaperlessTask, result_data: dict) -> None:
"""Record the scheduling outcome without completing an active parent."""
with transaction.atomic():
parent = PaperlessTask.objects.select_for_update().get(pk=task.pk)
if parent.is_complete:
return
parent.dispatched = True
parent.result_data = result_data
parent.save(update_fields=["dispatched", "result_data"])
_finalize_parent(parent)
def update_parent(task: PaperlessTask) -> None:
if task.parent_id is None:
return
with transaction.atomic():
parent = PaperlessTask.objects.select_for_update().get(pk=task.parent_id)
_finalize_parent(parent)
def _finalize_parent(parent: PaperlessTask) -> None:
if parent.is_complete or not parent.dispatched:
return
_schedule_finalizers(parent)
statuses = list(parent.children.values_list("status", flat=True))
if any(status not in PaperlessTask.COMPLETE_STATUSES for status in statuses):
return
succeeded = statuses.count(PaperlessTask.Status.SUCCESS)
failed = len(statuses) - succeeded
result_data = parent.result_data or {}
if failed and not result_data.get("error_message"):
result_data["error_message"] = (
f"{succeeded} child task(s) succeeded; {failed} failed or were revoked. "
"See child tasks for details."
)
parent.status = (
PaperlessTask.Status.FAILURE
if result_data.get("error_message")
else PaperlessTask.Status.SUCCESS
)
parent.result_data = {
**parent.input_data,
**result_data,
"succeeded_count": succeeded,
"failed_count": failed,
}
parent.date_done = timezone.now()
if parent.date_started:
parent.duration_seconds = (
parent.date_done - parent.date_started
).total_seconds()
parent.wait_time_seconds = (
parent.date_started - parent.date_created
).total_seconds()
parent.save()
from documents.signals.handlers import _send_bulk_operation_status
if "operation" in parent.input_data:
transaction.on_commit(lambda: _send_bulk_operation_status(parent))
update_parent(parent)
+3 -3
View File
@@ -394,11 +394,10 @@ def update_document_content_maybe_archive_file(
)
if not parser_class:
logger.error(
raise ValueError(
f"No parser found for mime type {mime_type}, cannot "
f"archive document {document} (ID: {document_id})",
f"reprocess document {document} (ID: {document_id})",
)
return
with parser_class() as parser:
parser.configure(ParserContext())
@@ -512,6 +511,7 @@ def update_document_content_maybe_archive_file(
logger.exception(
f"Error while parsing document {document} (ID: {document_id})",
)
raise
@shared_task
+2 -81
View File
@@ -4,18 +4,14 @@ from typing import TYPE_CHECKING
from unittest import mock
import pytest
from asgiref.sync import async_to_sync
from rest_framework import status
from rest_framework.test import APIClient
from rest_framework.test import APITestCase
from documents.views import _iterate_in_sync_thread
from paperless_testing.factories import UserFactory
from paperless_testing.permissions import grant_global
if TYPE_CHECKING:
from collections.abc import Generator
from django.contrib.auth.models import User
from pytest_mock import MockerFixture
@@ -66,7 +62,7 @@ class TestChatStreamingViewInputValidation(APITestCase):
),
mock.patch(
"paperless_ai.chat.stream_chat_with_documents",
return_value=(c for c in chunks),
return_value=iter(chunks),
),
):
resp = self.client.post(
@@ -78,11 +74,7 @@ class TestChatStreamingViewInputValidation(APITestCase):
assert resp.status_code == status.HTTP_200_OK
assert not resp.has_header("Content-Encoding")
async def read_parts() -> list[bytes]:
return [part async for part in resp.streaming_content]
assert async_to_sync(read_parts)() == [c.encode() for c in chunks]
assert list(resp.streaming_content) == [c.encode() for c in chunks]
def test_missing_question_is_rejected(self) -> None:
with mock.patch(
@@ -163,74 +155,3 @@ class TestChatStreamingViewUnrestrictedFlag:
assert (
mocked_stream_chat.call_args.kwargs["unrestricted"] is expected_unrestricted
)
class TestIterateInSyncThread:
@pytest.fixture
def events(self) -> list[str]:
return []
@pytest.fixture
def chunks(self, events: list[str]) -> Generator[str, None, None]:
def generate() -> Generator[str, None, None]:
try:
for chunk in ("first", "second"):
events.append(f"produced {chunk}")
yield chunk
finally:
events.append("closed")
return generate()
def test_yields_each_chunk_before_pulling_the_next(
self,
chunks: Generator[str, None, None],
events: list[str],
) -> None:
"""
GIVEN:
- A sync generator wrapped for async iteration
WHEN:
- The wrapper is consumed
THEN:
- Each chunk is delivered before the next one is generated,
rather than the whole generator being drained up front
"""
async def consume() -> None:
async for chunk in _iterate_in_sync_thread(chunks):
events.append(f"received {chunk}")
async_to_sync(consume)()
assert events == [
"produced first",
"received first",
"produced second",
"received second",
"closed",
]
def test_closing_early_closes_the_source_generator(
self,
chunks: Generator[str, None, None],
events: list[str],
) -> None:
"""
GIVEN:
- A sync generator wrapped for async iteration
WHEN:
- The consumer closes the wrapper after the first chunk, as
happens when the client disconnects
THEN:
- The source generator is closed, so its cleanup runs
"""
async def consume_one() -> None:
wrapper = _iterate_in_sync_thread(chunks)
await anext(wrapper)
await wrapper.aclose()
async_to_sync(consume_one)()
assert events == ["produced first", "closed"]
Loaded 100 of 164 files, more files were not shown because too many files have changed in this diff. Show more