mirror of
https://github.com/paperless-ngx/paperless-ngx.git
synced 2026-11-21 20:15:29 +00:00
Compare commits
29
Commits
| Author | SHA1 | Date | |
|---|---|---|---|
|
|
3508877a33 | ||
|
|
fea012d5c0 | ||
|
|
e2aa45da4e | ||
|
|
5ad0773446 | ||
|
|
13202f52ad | ||
|
|
9a9d4284d7 | ||
|
|
6043dd1c81 | ||
|
|
058e27e3ef | ||
|
|
6f5a192967 | ||
|
|
bb775a16c1 | ||
|
|
6d26d3f5ca | ||
|
|
2131feac73 | ||
|
|
e2705768b4 | ||
|
|
ce3c4e0855 | ||
|
|
3a241e51d9 | ||
|
|
020a2c8fc0 | ||
|
|
b533bfe4b5 | ||
|
|
813dfe487f | ||
|
|
557afa6454 | ||
|
|
c72473f7bc | ||
|
|
7d79b544ae | ||
|
|
a9fef2b931 | ||
|
|
adbd7561d4 | ||
|
|
9f694cdd2d | ||
|
|
7d78716069 | ||
|
|
9523d158a8 | ||
|
|
91316df9b7 | ||
|
|
7e67013c42 | ||
|
|
2a0c1249fa |
No files matched your search
+7
-7
@@ -285,14 +285,14 @@ The API supports various bulk-editing operations which are executed asynchronous
|
||||
|
||||
### Documents
|
||||
|
||||
Document bulk operations, including the dedicated document action endpoints, return
|
||||
`{"result":"OK","task_id":"…"}` when queued. Use `/api/tasks/?task_id={uuid}`
|
||||
to check the task outcome. Actions that create or reprocess files may start further
|
||||
consumption or reprocessing tasks and completion of the bulk task means those tasks
|
||||
were scheduled, not that their file processing has finished.
|
||||
Document bulk operations return `{"result":"OK","task_id":"…"}` when queued.
|
||||
Check `/api/tasks/?task_id={uuid}` for the outcome, including any follow-up processing,
|
||||
deletion, or recovery. Failed operations may leave partial changes applied.
|
||||
|
||||
For bulk operations on documents, use the endpoint `/api/documents/bulk_edit/` which accepts
|
||||
a json payload of the format:
|
||||
For operations with child tasks, `result_data` includes `succeeded_count` and
|
||||
`failed_count` (task counts). Merges report skipped inputs in `failed_document_ids`.
|
||||
|
||||
The `/api/documents/bulk_edit/` endpoint accepts:
|
||||
|
||||
```json
|
||||
{
|
||||
|
||||
@@ -1,121 +1,5 @@
|
||||
# Changelog
|
||||
|
||||
## paperless-ngx 3.5.1
|
||||
|
||||
### Changes
|
||||
|
||||
- Tweak: adjust chat timing for streaming [@shamoon](https://github.com/shamoon) ([#14805](https://github.com/paperless-ngx/paperless-ngx/pull/14805))
|
||||
|
||||
### Bug Fixes
|
||||
|
||||
- Fix: dont use set\_actor if audit log disabled in tasks [@shamoon](https://github.com/shamoon) ([#14808](https://github.com/paperless-ngx/paperless-ngx/pull/14808))
|
||||
- Fix: stream AI chat answers under ASGI [@stumpylog](https://github.com/stumpylog) ([#14803](https://github.com/paperless-ngx/paperless-ngx/pull/14803))
|
||||
- Tweak/fix: retain bulk edit selection after queuing operation [@shamoon](https://github.com/shamoon) ([#14797](https://github.com/paperless-ngx/paperless-ngx/pull/14797))
|
||||
|
||||
### All App Changes
|
||||
|
||||
<details>
|
||||
<summary>4 changes</summary>
|
||||
|
||||
- Fix: dont use set\_actor if audit log disabled in tasks [@shamoon](https://github.com/shamoon) ([#14808](https://github.com/paperless-ngx/paperless-ngx/pull/14808))
|
||||
- Tweak: adjust chat timing for streaming [@shamoon](https://github.com/shamoon) ([#14805](https://github.com/paperless-ngx/paperless-ngx/pull/14805))
|
||||
- Fix: stream AI chat answers under ASGI [@stumpylog](https://github.com/stumpylog) ([#14803](https://github.com/paperless-ngx/paperless-ngx/pull/14803))
|
||||
- Tweak/fix: retain bulk edit selection after queuing operation [@shamoon](https://github.com/shamoon) ([#14797](https://github.com/paperless-ngx/paperless-ngx/pull/14797))
|
||||
|
||||
</details>
|
||||
|
||||
## paperless-ngx 3.5.0
|
||||
|
||||
### Features / Enhancements
|
||||
|
||||
- Chorehancement: move all bulk operations to async hand-off [@shamoon](https://github.com/shamoon) ([#14773](https://github.com/paperless-ngx/paperless-ngx/pull/14773))
|
||||
- Enhancement: render document notes as markdown [@zwif](https://github.com/zwif) ([#14626](https://github.com/paperless-ngx/paperless-ngx/pull/14626))
|
||||
- Feature: manual workflow trigger [@shamoon](https://github.com/shamoon) ([#14704](https://github.com/paperless-ngx/paperless-ngx/pull/14704))
|
||||
- Enhancement: make bulk metadata operations run async [@shamoon](https://github.com/shamoon) ([#14711](https://github.com/paperless-ngx/paperless-ngx/pull/14711))
|
||||
- Fixhancement: add automatic clearing of sessions [@shamoon](https://github.com/shamoon) ([#14722](https://github.com/paperless-ngx/paperless-ngx/pull/14722))
|
||||
- Tweak: chat reference tweaks [@shamoon](https://github.com/shamoon) ([#14723](https://github.com/paperless-ngx/paperless-ngx/pull/14723))
|
||||
- Fixhancement: rework saved view widget queue processing to expected [@shamoon](https://github.com/shamoon) ([#14699](https://github.com/paperless-ngx/paperless-ngx/pull/14699))
|
||||
|
||||
### Bug Fixes
|
||||
|
||||
- Fix: use a constant password mask and consolidate the no-change check [@stumpylog](https://github.com/stumpylog) ([#14784](https://github.com/paperless-ngx/paperless-ngx/pull/14784))
|
||||
- Fix: use a PostgreSQL-compatible regex for scheduled workflow filename filters [@stumpylog](https://github.com/stumpylog) ([#14783](https://github.com/paperless-ngx/paperless-ngx/pull/14783))
|
||||
- Fix: get users for saved view create dialog [@shamoon](https://github.com/shamoon) ([#14769](https://github.com/paperless-ngx/paperless-ngx/pull/14769))
|
||||
- Fix: Allow AI chat filters over more than 32700 documents [@stumpylog](https://github.com/stumpylog) ([#14754](https://github.com/paperless-ngx/paperless-ngx/pull/14754))
|
||||
- Fix: Enforce CRLF when writing an email message to the file [@stumpylog](https://github.com/stumpylog) ([#14745](https://github.com/paperless-ngx/paperless-ngx/pull/14745))
|
||||
- Fix: restore image without archive version thumbnail generation [@shamoon](https://github.com/shamoon) ([#14741](https://github.com/paperless-ngx/paperless-ngx/pull/14741))
|
||||
- Fix: Blacklist certain OCR user arguments that paperless itself uses [@stumpylog](https://github.com/stumpylog) ([#14734](https://github.com/paperless-ngx/paperless-ngx/pull/14734))
|
||||
- Fix: remove redundant ui task text filters [@shamoon](https://github.com/shamoon) ([#14736](https://github.com/paperless-ngx/paperless-ngx/pull/14736))
|
||||
- Fix: fix allowed\_hosts default [@shamoon](https://github.com/shamoon) ([#14735](https://github.com/paperless-ngx/paperless-ngx/pull/14735))
|
||||
- Fixhancement: add automatic clearing of sessions [@shamoon](https://github.com/shamoon) ([#14722](https://github.com/paperless-ngx/paperless-ngx/pull/14722))
|
||||
- Fix: two dropdown-related UI fixes [@shamoon](https://github.com/shamoon) ([#14717](https://github.com/paperless-ngx/paperless-ngx/pull/14717))
|
||||
- Fix: use filtered documents for statistics ASN max [@shamoon](https://github.com/shamoon) ([#14710](https://github.com/paperless-ngx/paperless-ngx/pull/14710))
|
||||
- Fix: make config view permissions consistent [@shamoon](https://github.com/shamoon) ([#14709](https://github.com/paperless-ngx/paperless-ngx/pull/14709))
|
||||
- Fix: release the LLM index read lock before streaming the chat answer [@stumpylog](https://github.com/stumpylog) ([#14674](https://github.com/paperless-ngx/paperless-ngx/pull/14674))
|
||||
- Fix: retrieve once per document chat question [@stumpylog](https://github.com/stumpylog) ([#14673](https://github.com/paperless-ngx/paperless-ngx/pull/14673))
|
||||
- Fixhancement: rework saved view widget queue processing to expected [@shamoon](https://github.com/shamoon) ([#14699](https://github.com/paperless-ngx/paperless-ngx/pull/14699))
|
||||
- Fix: apply PDF/A conversion once for mail archives [@shamoon](https://github.com/shamoon) ([#14698](https://github.com/paperless-ngx/paperless-ngx/pull/14698))
|
||||
- Fix default MFA trusted AMR setting [@shamoon](https://github.com/shamoon) ([#14693](https://github.com/paperless-ngx/paperless-ngx/pull/14693))
|
||||
- Fix: check descendant tag permissions on delete [@shamoon](https://github.com/shamoon) ([#14687](https://github.com/paperless-ngx/paperless-ngx/pull/14687))
|
||||
- Fix: lock rows for first-user signup [@shamoon](https://github.com/shamoon) ([#14688](https://github.com/paperless-ngx/paperless-ngx/pull/14688))
|
||||
- Fix: prevent ID badge wrapping under title on mobile [@shamoon](https://github.com/shamoon) ([#14682](https://github.com/paperless-ngx/paperless-ngx/pull/14682))
|
||||
|
||||
### Maintenance
|
||||
|
||||
- Chore(deps): Bump the actions group across 1 directory with 17 updates @[dependabot[bot]](https://github.com/apps/dependabot) ([#14691](https://github.com/paperless-ngx/paperless-ngx/pull/14691))
|
||||
|
||||
### Dependencies
|
||||
|
||||
<details>
|
||||
<summary>4 changes</summary>
|
||||
|
||||
- Chore(deps): Bump the pre-commit-dependencies group with 4 updates @[dependabot[bot]](https://github.com/apps/dependabot) ([#14636](https://github.com/paperless-ngx/paperless-ngx/pull/14636))
|
||||
- Chore(deps): Bump the actions group across 1 directory with 17 updates @[dependabot[bot]](https://github.com/apps/dependabot) ([#14691](https://github.com/paperless-ngx/paperless-ngx/pull/14691))
|
||||
- docker(deps): bump astral-sh/uv from 0.13.5-python3.14-trixie-slim to 0.13.7-python3.14-trixie-slim @[dependabot[bot]](https://github.com/apps/dependabot) ([#14635](https://github.com/paperless-ngx/paperless-ngx/pull/14635))
|
||||
- chore(deps): bump django-allauth to 65.21.1 [@stumpylog](https://github.com/stumpylog) ([#14672](https://github.com/paperless-ngx/paperless-ngx/pull/14672))
|
||||
|
||||
</details>
|
||||
|
||||
### All App Changes
|
||||
|
||||
<details>
|
||||
<summary>32 changes</summary>
|
||||
|
||||
- Chorehancement: move all bulk operations to async hand-off [@shamoon](https://github.com/shamoon) ([#14773](https://github.com/paperless-ngx/paperless-ngx/pull/14773))
|
||||
- Fix: use a constant password mask and consolidate the no-change check [@stumpylog](https://github.com/stumpylog) ([#14784](https://github.com/paperless-ngx/paperless-ngx/pull/14784))
|
||||
- Fix: use a PostgreSQL-compatible regex for scheduled workflow filename filters [@stumpylog](https://github.com/stumpylog) ([#14783](https://github.com/paperless-ngx/paperless-ngx/pull/14783))
|
||||
- Fix: get users for saved view create dialog [@shamoon](https://github.com/shamoon) ([#14769](https://github.com/paperless-ngx/paperless-ngx/pull/14769))
|
||||
- Enhancement: render document notes as markdown [@zwif](https://github.com/zwif) ([#14626](https://github.com/paperless-ngx/paperless-ngx/pull/14626))
|
||||
- Feature: manual workflow trigger [@shamoon](https://github.com/shamoon) ([#14704](https://github.com/paperless-ngx/paperless-ngx/pull/14704))
|
||||
- Fix: Allow AI chat filters over more than 32700 documents [@stumpylog](https://github.com/stumpylog) ([#14754](https://github.com/paperless-ngx/paperless-ngx/pull/14754))
|
||||
- Fix: Enforce CRLF when writing an email message to the file [@stumpylog](https://github.com/stumpylog) ([#14745](https://github.com/paperless-ngx/paperless-ngx/pull/14745))
|
||||
- Enhancement: make bulk metadata operations run async [@shamoon](https://github.com/shamoon) ([#14711](https://github.com/paperless-ngx/paperless-ngx/pull/14711))
|
||||
- Performance: load a cached Hugging Face embedding model without contacting the Hub [@stumpylog](https://github.com/stumpylog) ([#14728](https://github.com/paperless-ngx/paperless-ngx/pull/14728))
|
||||
- Fix: restore image without archive version thumbnail generation [@shamoon](https://github.com/shamoon) ([#14741](https://github.com/paperless-ngx/paperless-ngx/pull/14741))
|
||||
- Fix: Blacklist certain OCR user arguments that paperless itself uses [@stumpylog](https://github.com/stumpylog) ([#14734](https://github.com/paperless-ngx/paperless-ngx/pull/14734))
|
||||
- Fix: remove redundant ui task text filters [@shamoon](https://github.com/shamoon) ([#14736](https://github.com/paperless-ngx/paperless-ngx/pull/14736))
|
||||
- Fix: fix allowed\_hosts default [@shamoon](https://github.com/shamoon) ([#14735](https://github.com/paperless-ngx/paperless-ngx/pull/14735))
|
||||
- Fixhancement: add automatic clearing of sessions [@shamoon](https://github.com/shamoon) ([#14722](https://github.com/paperless-ngx/paperless-ngx/pull/14722))
|
||||
- Tweak: chat reference tweaks [@shamoon](https://github.com/shamoon) ([#14723](https://github.com/paperless-ngx/paperless-ngx/pull/14723))
|
||||
- Fix: two dropdown-related UI fixes [@shamoon](https://github.com/shamoon) ([#14717](https://github.com/paperless-ngx/paperless-ngx/pull/14717))
|
||||
- Fix: use filtered documents for statistics ASN max [@shamoon](https://github.com/shamoon) ([#14710](https://github.com/paperless-ngx/paperless-ngx/pull/14710))
|
||||
- Fix: make config view permissions consistent [@shamoon](https://github.com/shamoon) ([#14709](https://github.com/paperless-ngx/paperless-ngx/pull/14709))
|
||||
- Performance: stop embedding every document ID in selection\_data queries [@stumpylog](https://github.com/stumpylog) ([#14597](https://github.com/paperless-ngx/paperless-ngx/pull/14597))
|
||||
- Performance: replace django-treenode with adjacency-list tags [@shamoon](https://github.com/shamoon) ([#14612](https://github.com/paperless-ngx/paperless-ngx/pull/14612))
|
||||
- Performance: use small vec0 chunks and larger pages for the LLM index [@stumpylog](https://github.com/stumpylog) ([#14678](https://github.com/paperless-ngx/paperless-ngx/pull/14678))
|
||||
- Fix: release the LLM index read lock before streaming the chat answer [@stumpylog](https://github.com/stumpylog) ([#14674](https://github.com/paperless-ngx/paperless-ngx/pull/14674))
|
||||
- Fix: retrieve once per document chat question [@stumpylog](https://github.com/stumpylog) ([#14673](https://github.com/paperless-ngx/paperless-ngx/pull/14673))
|
||||
- Fixhancement: rework saved view widget queue processing to expected [@shamoon](https://github.com/shamoon) ([#14699](https://github.com/paperless-ngx/paperless-ngx/pull/14699))
|
||||
- Fix: apply PDF/A conversion once for mail archives [@shamoon](https://github.com/shamoon) ([#14698](https://github.com/paperless-ngx/paperless-ngx/pull/14698))
|
||||
- Fix default MFA trusted AMR setting [@shamoon](https://github.com/shamoon) ([#14693](https://github.com/paperless-ngx/paperless-ngx/pull/14693))
|
||||
- Fix: check descendant tag permissions on delete [@shamoon](https://github.com/shamoon) ([#14687](https://github.com/paperless-ngx/paperless-ngx/pull/14687))
|
||||
- Fix: lock rows for first-user signup [@shamoon](https://github.com/shamoon) ([#14688](https://github.com/paperless-ngx/paperless-ngx/pull/14688))
|
||||
- Fix: prevent ID badge wrapping under title on mobile [@shamoon](https://github.com/shamoon) ([#14682](https://github.com/paperless-ngx/paperless-ngx/pull/14682))
|
||||
- chore(deps): bump django-allauth to 65.21.1 [@stumpylog](https://github.com/stumpylog) ([#14672](https://github.com/paperless-ngx/paperless-ngx/pull/14672))
|
||||
- chore(deps): bump dev tooling [@stumpylog](https://github.com/stumpylog) ([#14671](https://github.com/paperless-ngx/paperless-ngx/pull/14671))
|
||||
|
||||
</details>
|
||||
|
||||
## paperless-ngx 3.4.2
|
||||
|
||||
### Bug Fixes
|
||||
|
||||
+3
-4
@@ -1,6 +1,6 @@
|
||||
[project]
|
||||
name = "paperless-ngx"
|
||||
version = "3.5.1"
|
||||
version = "3.4.2"
|
||||
description = """\
|
||||
A community-supported supercharged document management system: scan, index and archive all your physical documents\
|
||||
"""
|
||||
@@ -151,9 +151,8 @@ typing = [
|
||||
required-version = ">=0.9.0"
|
||||
prerelease = "disallow"
|
||||
environments = [
|
||||
"sys_platform == 'darwin' and platform_machine == 'arm64' and platform_python_implementation == 'CPython'",
|
||||
"sys_platform == 'linux' and platform_machine == 'aarch64' and platform_python_implementation == 'CPython'",
|
||||
"sys_platform == 'linux' and platform_machine == 'x86_64' and platform_python_implementation == 'CPython'",
|
||||
"sys_platform == 'darwin'",
|
||||
"sys_platform == 'linux'",
|
||||
]
|
||||
package = false
|
||||
[tool.uv.sources]
|
||||
|
||||
+249
-490
File diff suppressed because it is too large.
Load diff
+1
-1
@@ -1,6 +1,6 @@
|
||||
{
|
||||
"name": "paperless-ngx-ui",
|
||||
"version": "3.5.1",
|
||||
"version": "3.4.2",
|
||||
"scripts": {
|
||||
"preinstall": "npx only-allow pnpm",
|
||||
"ng": "ng",
|
||||
|
||||
@@ -227,12 +227,6 @@
|
||||
</div>
|
||||
</div>
|
||||
|
||||
<div class="row">
|
||||
<div class="col">
|
||||
<pngx-input-check i18n-title title="Show nested tags with their parents" i18n-hint hint="Child tags are shown together with their parent tags, instead of separately." formControlName="tagsShowHierarchy"></pngx-input-check>
|
||||
</div>
|
||||
</div>
|
||||
|
||||
<h5 class="mt-3" i18n>Document editing</h5>
|
||||
<div class="row">
|
||||
<div class="col">
|
||||
|
||||
@@ -305,7 +305,7 @@ describe('SettingsComponent', () => {
|
||||
expect(toastErrorSpy).toHaveBeenCalled()
|
||||
expect(storeSpy).toHaveBeenCalled()
|
||||
expect(appearanceSettingsSpy).not.toHaveBeenCalled()
|
||||
expect(setSpy).toHaveBeenCalledTimes(34)
|
||||
expect(setSpy).toHaveBeenCalledTimes(33)
|
||||
expect(setSpy).toHaveBeenCalledWith(SETTINGS_KEYS.SIDEBAR_HIDDEN_ITEMS, [
|
||||
HideableSidebarItemID.Workflows,
|
||||
])
|
||||
|
||||
@@ -163,7 +163,6 @@ export class SettingsComponent
|
||||
bulkEditConfirmationDialogs: new FormControl(null),
|
||||
bulkEditApplyOnClose: new FormControl(null),
|
||||
documentListItemPerPage: new FormControl(null),
|
||||
tagsShowHierarchy: new FormControl(null),
|
||||
slimSidebarEnabled: new FormControl(null),
|
||||
sidebarHiddenItems: new FormControl<HideableSidebarItemID[]>([]),
|
||||
darkModeUseSystem: new FormControl(null),
|
||||
@@ -343,7 +342,6 @@ export class SettingsComponent
|
||||
bulkEditApplyOnClose: this.settings.get(
|
||||
SETTINGS_KEYS.BULK_EDIT_APPLY_ON_CLOSE
|
||||
),
|
||||
tagsShowHierarchy: this.settings.get(SETTINGS_KEYS.TAGS_SHOW_HIERARCHY),
|
||||
documentListItemPerPage: this.settings.get(
|
||||
SETTINGS_KEYS.DOCUMENT_LIST_SIZE
|
||||
),
|
||||
@@ -609,10 +607,6 @@ export class SettingsComponent
|
||||
SETTINGS_KEYS.DEFAULT_PERMS_EDIT_GROUPS,
|
||||
this.settingsForm.value.defaultPermsEditGroups
|
||||
)
|
||||
this.settings.set(
|
||||
SETTINGS_KEYS.TAGS_SHOW_HIERARCHY,
|
||||
this.settingsForm.value.tagsShowHierarchy
|
||||
)
|
||||
this.settings.set(
|
||||
SETTINGS_KEYS.DOCUMENT_EDITING_REMOVE_INBOX_TAGS,
|
||||
this.settingsForm.value.documentEditingRemoveInboxTags
|
||||
|
||||
@@ -186,9 +186,7 @@ export class AppFrameComponent
|
||||
)
|
||||
.subscribe((result) => {
|
||||
this.documentList.reload(null, false)
|
||||
if (!this.documentList.allSelected) {
|
||||
this.documentList.reduceSelectionToFilter()
|
||||
}
|
||||
this.documentList.reduceSelectionToFilter()
|
||||
// maybe someday we can optimize to only refresh affected documents
|
||||
this.openDocumentsService.getOpenDocuments().forEach((doc) => {
|
||||
this.openDocumentsService.refreshDocument(doc.id)
|
||||
|
||||
@@ -188,62 +188,6 @@ describe('ChatComponent', () => {
|
||||
expect(component.loading()).toBe(false)
|
||||
})
|
||||
|
||||
it('should reveal a complete answer within one second', () => {
|
||||
component.input.set('Hello')
|
||||
component.sendMessage()
|
||||
|
||||
const answer = 'A'.repeat(660)
|
||||
mockStream$.next(answer)
|
||||
mockStream$.complete()
|
||||
|
||||
expect(component.messages()[1].content.length).toBeGreaterThan(0)
|
||||
expect(component.messages()[1].content.length).toBeLessThan(answer.length)
|
||||
expect(component.loading()).toBe(true)
|
||||
|
||||
jest.advanceTimersByTime(800)
|
||||
expect(component.messages()[1].content).toBe(answer)
|
||||
expect(component.loading()).toBe(false)
|
||||
})
|
||||
|
||||
it('should give a long answer more time to reveal', () => {
|
||||
component.input.set('Hello')
|
||||
component.sendMessage()
|
||||
|
||||
const answer = 'A'.repeat(3000)
|
||||
mockStream$.next(answer)
|
||||
mockStream$.complete()
|
||||
|
||||
jest.advanceTimersByTime(800)
|
||||
expect(component.messages()[1].content.length).toBeLessThan(answer.length)
|
||||
expect(component.loading()).toBe(true)
|
||||
|
||||
jest.advanceTimersByTime(2500)
|
||||
expect(component.messages()[1].content).toBe(answer)
|
||||
expect(component.loading()).toBe(false)
|
||||
})
|
||||
|
||||
it('should reveal fast incoming chunks without making them pop in', () => {
|
||||
component.input.set('Hello')
|
||||
component.sendMessage()
|
||||
|
||||
let answer = ''
|
||||
for (let i = 0; i < 20; i++) {
|
||||
answer += 'A'.repeat(100)
|
||||
const visibleBefore = component.messages()[1].content.length
|
||||
mockStream$.next(answer)
|
||||
jest.advanceTimersByTime(10)
|
||||
expect(
|
||||
component.messages()[1].content.length - visibleBefore
|
||||
).toBeLessThan(100)
|
||||
}
|
||||
expect(component.messages()[1].content.length).toBeLessThan(answer.length)
|
||||
|
||||
mockStream$.complete()
|
||||
jest.advanceTimersByTime(5200)
|
||||
expect(component.messages()[1].content).toBe(answer)
|
||||
expect(component.loading()).toBe(false)
|
||||
})
|
||||
|
||||
it('should show a slow response message while waiting', () => {
|
||||
component.input.set('Hello')
|
||||
component.sendMessage()
|
||||
|
||||
@@ -21,9 +21,6 @@ import {
|
||||
import { pngxPopperOptions } from 'src/app/utils/popper-options'
|
||||
|
||||
const SLOW_RESPONSE_DELAY_MS = 8000
|
||||
const TYPEWRITER_MIN_DURATION_MS = 750
|
||||
const TYPEWRITER_MAX_DURATION_MS = 3000
|
||||
const TYPEWRITER_INTERVAL_MS = 16
|
||||
|
||||
@Component({
|
||||
selector: 'pngx-chat',
|
||||
@@ -76,7 +73,6 @@ export class ChatComponent implements OnInit {
|
||||
private typewriterBuffer: string[] = []
|
||||
private typewriterActive = false
|
||||
private typewriterTimeout: ReturnType<typeof setTimeout>
|
||||
private typewriterDeadline = 0
|
||||
private streamSubscription: Subscription
|
||||
private streamingMessage: ChatMessage
|
||||
private streamFinished = false
|
||||
@@ -138,7 +134,7 @@ export class ChatComponent implements OnInit {
|
||||
if (nextResponse.content.length < lastVisibleContent.length) {
|
||||
this.resetTypewriter(assistantMessage, nextResponse.content)
|
||||
lastVisibleContent = nextResponse.content
|
||||
} else if (nextResponse.content.length > lastVisibleContent.length) {
|
||||
} else {
|
||||
const visibleDelta = nextResponse.content.substring(
|
||||
lastVisibleContent.length
|
||||
)
|
||||
@@ -218,12 +214,6 @@ export class ChatComponent implements OnInit {
|
||||
if (!chunk) return
|
||||
|
||||
this.typewriterBuffer.push(...chunk.split(''))
|
||||
// Longer bursts need more time to remain readable as the chat scrolls.
|
||||
const revealDuration = Math.min(
|
||||
TYPEWRITER_MAX_DURATION_MS,
|
||||
Math.max(TYPEWRITER_MIN_DURATION_MS, this.typewriterBuffer.length)
|
||||
)
|
||||
this.typewriterDeadline = Date.now() + revealDuration
|
||||
|
||||
if (!this.typewriterActive) {
|
||||
this.typewriterActive = true
|
||||
@@ -232,29 +222,19 @@ export class ChatComponent implements OnInit {
|
||||
}
|
||||
|
||||
playTypewriter(message: ChatMessage): void {
|
||||
// Spread the queued text across the updates left before the deadline
|
||||
const remainingUpdates = Math.max(
|
||||
1,
|
||||
Math.ceil((this.typewriterDeadline - Date.now()) / TYPEWRITER_INTERVAL_MS)
|
||||
)
|
||||
const charactersToShow = Math.ceil(
|
||||
this.typewriterBuffer.length / remainingUpdates
|
||||
)
|
||||
message.content += this.typewriterBuffer
|
||||
.splice(0, charactersToShow)
|
||||
.join('')
|
||||
this.notifyMessagesChanged()
|
||||
this.scrollToBottom()
|
||||
|
||||
if (this.typewriterBuffer.length === 0) {
|
||||
this.typewriterActive = false
|
||||
if (this.streamFinished) this.finishStreaming(message)
|
||||
} else {
|
||||
this.typewriterTimeout = setTimeout(
|
||||
() => this.playTypewriter(message),
|
||||
TYPEWRITER_INTERVAL_MS
|
||||
)
|
||||
return
|
||||
}
|
||||
|
||||
const nextChar = this.typewriterBuffer.shift()
|
||||
message.content += nextChar
|
||||
this.notifyMessagesChanged()
|
||||
this.scrollToBottom()
|
||||
|
||||
// 10ms per character
|
||||
this.typewriterTimeout = setTimeout(() => this.playTypewriter(message), 10)
|
||||
}
|
||||
|
||||
private notifyMessagesChanged(): void {
|
||||
|
||||
@@ -1,90 +0,0 @@
|
||||
import { provideHttpClient } from '@angular/common/http'
|
||||
import { provideHttpClientTesting } from '@angular/common/http/testing'
|
||||
import { Component } from '@angular/core'
|
||||
import { ComponentFixture, TestBed } from '@angular/core/testing'
|
||||
import { FormControl, ReactiveFormsModule } from '@angular/forms'
|
||||
import { NumberComponent } from './number/number.component'
|
||||
import { TextComponent } from './text/text.component'
|
||||
import { TextAreaComponent } from './textarea/textarea.component'
|
||||
import { UrlComponent } from './url/url.component'
|
||||
|
||||
@Component({
|
||||
imports: [
|
||||
ReactiveFormsModule,
|
||||
TextComponent,
|
||||
NumberComponent,
|
||||
TextAreaComponent,
|
||||
UrlComponent,
|
||||
],
|
||||
template: `
|
||||
<pngx-input-text [formControl]="control" />
|
||||
<pngx-input-number [formControl]="control" [showAdd]="false" />
|
||||
<pngx-input-textarea [formControl]="control" />
|
||||
<pngx-input-url [formControl]="control" />
|
||||
`,
|
||||
})
|
||||
class InputEventsHostComponent {
|
||||
control = new FormControl<string | number | null>(null)
|
||||
}
|
||||
|
||||
describe.each([
|
||||
['text', 'pngx-input-text input', 'Edited text', 'Edited text'],
|
||||
['number', 'pngx-input-number input', '42', 42],
|
||||
['textarea', 'pngx-input-textarea textarea', 'Edited\ntext', 'Edited\ntext'],
|
||||
['url', 'pngx-input-url input', 'https://example.com', 'https://example.com'],
|
||||
])(
|
||||
'%s input events',
|
||||
(_type, selector: string, text: string, value: string | number) => {
|
||||
let fixture: ComponentFixture<InputEventsHostComponent>
|
||||
let control: FormControl<string | number | null>
|
||||
let input: HTMLInputElement | HTMLTextAreaElement
|
||||
|
||||
beforeEach(() => {
|
||||
TestBed.configureTestingModule({
|
||||
imports: [InputEventsHostComponent],
|
||||
providers: [provideHttpClient(), provideHttpClientTesting()],
|
||||
})
|
||||
fixture = TestBed.createComponent(InputEventsHostComponent)
|
||||
control = fixture.componentInstance.control
|
||||
})
|
||||
|
||||
async function render() {
|
||||
fixture.detectChanges()
|
||||
await fixture.whenStable()
|
||||
input = fixture.nativeElement.querySelector(selector)
|
||||
}
|
||||
|
||||
it('updates the form while typing and marks it touched only on blur', async () => {
|
||||
await render()
|
||||
const changed = jest.fn()
|
||||
control.valueChanges.subscribe(changed)
|
||||
input.value = text
|
||||
input.dispatchEvent(new Event('input'))
|
||||
expect(control.value).toBe(value)
|
||||
expect(control.dirty).toBe(true)
|
||||
expect(control.touched).toBe(false)
|
||||
|
||||
input.dispatchEvent(new Event('change'))
|
||||
input.dispatchEvent(new Event('blur'))
|
||||
expect(control.touched).toBe(true)
|
||||
expect(changed).toHaveBeenCalledTimes(1)
|
||||
})
|
||||
|
||||
it('honors an explicit updateOn blur strategy', async () => {
|
||||
control = new FormControl<string | number | null>(null, {
|
||||
updateOn: 'blur',
|
||||
})
|
||||
fixture.componentInstance.control = control
|
||||
await render()
|
||||
input.value = text
|
||||
input.dispatchEvent(new Event('input'))
|
||||
expect(control.value).toBeNull()
|
||||
expect(control.pristine).toBe(true)
|
||||
|
||||
input.dispatchEvent(new Event('blur'))
|
||||
expect(control.value).toBe(value)
|
||||
expect(control.dirty).toBe(true)
|
||||
expect(control.touched).toBe(true)
|
||||
})
|
||||
}
|
||||
)
|
||||
@@ -12,7 +12,7 @@
|
||||
</div>
|
||||
<div class="position-relative" [class.col-md-9]="horizontal">
|
||||
<div class="input-group" [class.is-invalid]="error">
|
||||
<input #inputField type="number" class="form-control" [step]="step" [id]="inputId" [(ngModel)]="value" [ngModelOptions]="{standalone: true}" (ngModelChange)="onChange($event)" (blur)="onTouched()" [class.is-invalid]="error" [disabled]="disabled">
|
||||
<input #inputField type="number" class="form-control" [step]="step" [id]="inputId" [(ngModel)]="value" [ngModelOptions]="{standalone: true}" (change)="onChange(value)" [class.is-invalid]="error" [disabled]="disabled">
|
||||
@if (showAdd) {
|
||||
<button class="btn btn-outline-secondary" type="button" id="button-addon1" (click)="nextAsn()" [disabled]="disabled">+1</button>
|
||||
}
|
||||
|
||||
@@ -17,21 +17,16 @@
|
||||
[searchFn]="searchFn"
|
||||
addTagText="Add tag"
|
||||
i18n-addTagText
|
||||
[keyDownFn]="onKeyDown"
|
||||
(add)="onAdd($event)"
|
||||
(change)="onChange(value)">
|
||||
|
||||
<ng-template ng-label-tmp let-item="item">
|
||||
@if (isImplied(item.id)) {
|
||||
<span class="implied-tag"></span>
|
||||
} @else {
|
||||
<button class="tag-wrap btn p-0 d-flex align-items-center mw-100" (click)="removeBadge(item.id)" (mousedown)="$event.stopImmediatePropagation()" type="button" title="Remove tag" i18n-title>
|
||||
<pngx-bs-icon name="x" style="margin-inline-end: 1px;"></pngx-bs-icon>
|
||||
@if (item.id && tags) {
|
||||
<pngx-tag class="mw-100 overflow-hidden" [tag]="getTag(item.id)" [split]="preserveHierarchy" (click)="removeTag(item.id); $event.stopPropagation()" (parentClick)="removeTag($event)"></pngx-tag>
|
||||
}
|
||||
</button>
|
||||
}
|
||||
<button class="tag-wrap btn p-0 d-flex align-items-center" (click)="removeTag(item.id)" (mousedown)="$event.stopImmediatePropagation()" type="button" title="Remove tag" i18n-title>
|
||||
<pngx-bs-icon name="x" style="margin-inline-end: 1px;"></pngx-bs-icon>
|
||||
@if (item.id && tags) {
|
||||
<pngx-tag style="background-color: none;" [tag]="getTag(item.id)"></pngx-tag>
|
||||
}
|
||||
</button>
|
||||
</ng-template>
|
||||
<ng-template ng-option-tmp let-item="item" let-index="index" let-search="searchTerm">
|
||||
<div class="tag-option-row d-flex align-items-center" [class.w-auto]="!getTag(item.id)?.parent">
|
||||
|
||||
@@ -7,11 +7,6 @@
|
||||
font-size: 1rem;
|
||||
}
|
||||
|
||||
// ancestors of another selected tag render inside that tag's badge
|
||||
::ng-deep .paperless-input-tags .ng-value:has(> .implied-tag) {
|
||||
display: none !important;
|
||||
}
|
||||
|
||||
.paperless-input-select.disabled {
|
||||
.input-group {
|
||||
cursor: not-allowed;
|
||||
|
||||
@@ -278,59 +278,4 @@ describe('TagsComponent', () => {
|
||||
component.tags = [lone]
|
||||
expect(component.getParentChain(5)).toEqual([])
|
||||
})
|
||||
|
||||
describe('hierarchy', () => {
|
||||
const root: Tag = { id: 1, name: 'root' }
|
||||
const mid: Tag = { id: 2, name: 'mid', parent: 1 }
|
||||
const leaf: Tag = { id: 3, name: 'leaf', parent: 2 }
|
||||
const other: Tag = { id: 4, name: 'other' }
|
||||
|
||||
beforeEach(() => {
|
||||
component.tags = [root, mid, leaf, other]
|
||||
component.value = [3, 2, 1, 4]
|
||||
})
|
||||
|
||||
it('should treat ancestors of another selected tag as implied', () => {
|
||||
expect([1, 2, 3, 4].map((id) => component.isImplied(id))).toEqual([
|
||||
true,
|
||||
true,
|
||||
false,
|
||||
false,
|
||||
])
|
||||
component.preserveHierarchy = false
|
||||
expect(component.isImplied(1)).toBe(false)
|
||||
})
|
||||
|
||||
it('should remove a whole badge, keeping ancestors other tags still need', () => {
|
||||
component.removeBadge(3)
|
||||
expect(component.value).toEqual([4])
|
||||
|
||||
const sibling: Tag = { id: 5, name: 'sibling', parent: 1 }
|
||||
component.tags = [root, mid, leaf, other, sibling]
|
||||
component.value = [3, 2, 1, 5, 4]
|
||||
component.removeBadge(3)
|
||||
expect(component.value).toEqual([1, 5, 4])
|
||||
})
|
||||
|
||||
it('should remove only the tag itself from a badge when hierarchy is not shown', () => {
|
||||
component.preserveHierarchy = false
|
||||
component.removeBadge(3)
|
||||
expect(component.value).toEqual([2, 1, 4])
|
||||
})
|
||||
|
||||
it('should remove the last visible tag on backspace, not a hidden ancestor', () => {
|
||||
const backspace = new KeyboardEvent('keydown', { key: 'Backspace' })
|
||||
expect(component.onKeyDown(backspace)).toBe(false)
|
||||
expect(component.value).toEqual([3, 2, 1])
|
||||
expect(component.onKeyDown(backspace)).toBe(false)
|
||||
expect(component.value).toEqual([2, 1])
|
||||
|
||||
// nothing hidden: leave it to ng-select
|
||||
component.value = [4]
|
||||
expect(component.onKeyDown(backspace)).toBe(true)
|
||||
expect(
|
||||
component.onKeyDown(new KeyboardEvent('keydown', { key: 'a' }))
|
||||
).toBe(true)
|
||||
})
|
||||
})
|
||||
})
|
||||
@@ -21,13 +21,7 @@ import { NgSelectComponent, NgSelectModule } from '@ng-select/ng-select'
|
||||
import { first, firstValueFrom, tap } from 'rxjs'
|
||||
import { BootstrapIconComponent } from 'src/app/components/common/bootstrap-icon/bootstrap-icon.component'
|
||||
import { Tag } from 'src/app/data/tag'
|
||||
import { SETTINGS_KEYS } from 'src/app/data/ui-settings'
|
||||
import { TagService } from 'src/app/services/rest/tag.service'
|
||||
import { SettingsService } from 'src/app/services/settings.service'
|
||||
import {
|
||||
collapseImpliedAncestors,
|
||||
getTagAncestors,
|
||||
} from 'src/app/utils/collapse-tags'
|
||||
import { matchesSearchText } from 'src/app/utils/text-search'
|
||||
import { EditDialogMode } from '../../edit-dialog/edit-dialog.component'
|
||||
import { TagEditDialogComponent } from '../../edit-dialog/tag-edit-dialog/tag-edit-dialog.component'
|
||||
@@ -56,7 +50,6 @@ import { TagComponent } from '../../tag/tag.component'
|
||||
export class TagsComponent implements OnInit, ControlValueAccessor {
|
||||
private tagService = inject(TagService)
|
||||
private modalService = inject(NgbModal)
|
||||
private readonly settingsService = inject(SettingsService)
|
||||
private readonly changeDetector = inject(ChangeDetectorRef)
|
||||
|
||||
constructor() {
|
||||
@@ -138,33 +131,6 @@ export class TagsComponent implements OnInit, ControlValueAccessor {
|
||||
term
|
||||
)
|
||||
|
||||
/**
|
||||
* Selected tags that are ancestors of another selected tag which are shown
|
||||
* as segments of their descendant's badge.
|
||||
*/
|
||||
isImplied(id: number): boolean {
|
||||
if (!this.showsHierarchy || !this.tags || !this.value) return false
|
||||
return !collapseImpliedAncestors(this.value, this.tagsById).includes(id)
|
||||
}
|
||||
|
||||
private get showsHierarchy(): boolean {
|
||||
return (
|
||||
this.preserveHierarchy &&
|
||||
!!this.settingsService.get(SETTINGS_KEYS.TAGS_SHOW_HIERARCHY)
|
||||
)
|
||||
}
|
||||
|
||||
/**
|
||||
* Remove the last visible tag (in case its actually a hidden ancestor)
|
||||
*/
|
||||
readonly onKeyDown = (event: KeyboardEvent): boolean => {
|
||||
if (event.key !== 'Backspace' || this.select?.searchTerm) return true
|
||||
const visible = this.value?.filter((id) => !this.isImplied(id)) ?? []
|
||||
if (visible.length === (this.value?.length ?? 0)) return true
|
||||
this.removeTag(visible.at(-1))
|
||||
return false
|
||||
}
|
||||
|
||||
getTag(id: number) {
|
||||
if (this.tags) {
|
||||
return this.tags.find((tag) => tag.id == id)
|
||||
@@ -173,26 +139,6 @@ export class TagsComponent implements OnInit, ControlValueAccessor {
|
||||
}
|
||||
}
|
||||
|
||||
/**
|
||||
* Remove a whole badge i.e. the tag plus any ancestors that no other selected
|
||||
* tag still needs.
|
||||
*/
|
||||
removeBadge(tagID: number) {
|
||||
if (this.disabled) return
|
||||
if (!this.showsHierarchy) return this.removeTag(tagID)
|
||||
|
||||
const ancestors = new Set(this.getParentChain(tagID).map((t) => t.id))
|
||||
const needed = new Set(
|
||||
this.value
|
||||
.filter((id) => id !== tagID && !ancestors.has(id))
|
||||
.flatMap((id) => this.getParentChain(id).map((t) => t.id))
|
||||
)
|
||||
this.value = this.value.filter(
|
||||
(id) => id !== tagID && (!ancestors.has(id) || needed.has(id))
|
||||
)
|
||||
this.onChange(this.value)
|
||||
}
|
||||
|
||||
removeTag(tagID: number) {
|
||||
if (this.disabled) return
|
||||
|
||||
@@ -289,17 +235,18 @@ export class TagsComponent implements OnInit, ControlValueAccessor {
|
||||
}
|
||||
|
||||
getParentChain(id: number): Tag[] {
|
||||
return getTagAncestors(id, this.tagsById)
|
||||
}
|
||||
|
||||
private tagsByIdSource: Tag[]
|
||||
private tagsByIdCache: Map<number, Tag>
|
||||
|
||||
private get tagsById(): Map<number, Tag> {
|
||||
if (this.tagsByIdSource !== this.tags) {
|
||||
this.tagsByIdSource = this.tags
|
||||
this.tagsByIdCache = new Map(this.tags?.map((t) => [t.id, t]))
|
||||
// Returns ancestors from root → immediate parent for a tag id
|
||||
const chain: Tag[] = []
|
||||
let current = this.getTag(id)
|
||||
const guard = new Set<number>()
|
||||
while (current?.parent) {
|
||||
if (guard.has(current.parent)) break
|
||||
guard.add(current.parent)
|
||||
const parent = this.getTag(current.parent)
|
||||
if (!parent) break
|
||||
chain.unshift(parent)
|
||||
current = parent
|
||||
}
|
||||
return this.tagsByIdCache
|
||||
return chain
|
||||
}
|
||||
}
|
||||
@@ -11,7 +11,7 @@
|
||||
}
|
||||
</div>
|
||||
<div class="position-relative" [class.col-md-9]="horizontal">
|
||||
<input #inputField type="text" class="form-control" [class.is-invalid]="error" [id]="inputId" [(ngModel)]="value" [ngModelOptions]="{standalone: true}" (ngModelChange)="onChange($event)" (blur)="onTouched()" [disabled]="disabled" [autocomplete]="autocomplete" [placeholder]="placeholder">
|
||||
<input #inputField type="text" class="form-control" [class.is-invalid]="error" [id]="inputId" [(ngModel)]="value" [ngModelOptions]="{standalone: true}" (change)="onChange(value)" [disabled]="disabled" [autocomplete]="autocomplete" [placeholder]="placeholder">
|
||||
@if (hint) {
|
||||
<small class="form-text text-muted" [innerHTML]="hint"></small>
|
||||
}
|
||||
|
||||
@@ -18,8 +18,7 @@
|
||||
[class.font-monospace]="monospace"
|
||||
[(ngModel)]="value"
|
||||
[ngModelOptions]="{standalone: true}"
|
||||
(ngModelChange)="onChange($event)"
|
||||
(blur)="onTouched()"
|
||||
(change)="onChange(value)"
|
||||
[disabled]="disabled"
|
||||
[placeholder]="placeholder"
|
||||
rows="4">
|
||||
|
||||
@@ -10,7 +10,7 @@
|
||||
</div>
|
||||
<div [class.col-md-9]="horizontal">
|
||||
<div class="input-group" [class.is-invalid]="error">
|
||||
<input #inputField type="url" class="form-control" [class.is-invalid]="error" placeholder="https://" [id]="inputId" [(ngModel)]="value" [ngModelOptions]="{standalone: true}" (ngModelChange)="onChange($event)" (blur)="onTouched()" [disabled]="disabled">
|
||||
<input #inputField type="url" class="form-control" [class.is-invalid]="error" placeholder="https://" [id]="inputId" [(ngModel)]="value" [ngModelOptions]="{standalone: true}" (change)="onChange(value)" [disabled]="disabled">
|
||||
<a class="btn btn-outline-secondary rounded-end" title="Open link" i18n-title [href]="value" target="_blank">
|
||||
<pngx-bs-icon width="1.2em" height="1.2em" name="box-arrow-up-right"></pngx-bs-icon>
|
||||
</a>
|
||||
|
||||
@@ -1,18 +1,13 @@
|
||||
@if (tag()) {
|
||||
@if (ancestorSegments.length) {
|
||||
<span #pill class="split-pill d-inline-flex mw-100" [title]="path()" (mouseleave)="setFocus(pill, null)">
|
||||
@for (ancestor of ancestorSegments; track ancestor.id; let first = $first) {
|
||||
<span #segment class="badge split-parent" [class.rounded-end-0]="first" [class.rounded-0]="!first" [style.background]="ancestor.color" [style.color]="ancestor.text_color" (mouseenter)="setFocus(pill, segment)" (click)="onParentClick($event, ancestor.id)">{{ancestor.name}}</span>
|
||||
}
|
||||
<span #leaf class="badge rounded-start-0 split-child" [style.background]="tag().color" [style.color]="tag().text_color" (mouseenter)="setFocus(pill, leaf)">{{tag().name}}</span>
|
||||
</span>
|
||||
} @else {
|
||||
@if (!clickable()) {
|
||||
<span class="badge" [style.background]="tag().color" [style.color]="tag().text_color">{{tag().name}}</span>
|
||||
}
|
||||
@if (clickable()) {
|
||||
<a [title]="linkTitle()" class="badge" [style.background]="tag().color" [style.color]="tag().text_color">{{tag().name}}</a>
|
||||
}
|
||||
@if (showParents() && tag().parent) {
|
||||
<pngx-tag [tagID]="tag().parent" [clickable]="clickable()" [linkTitle]="linkTitle()"></pngx-tag>
|
||||
>
|
||||
}
|
||||
@if (!clickable()) {
|
||||
<span class="badge" [style.background]="tag().color" [style.color]="tag().text_color">{{tag().name}}</span>
|
||||
}
|
||||
@if (clickable()) {
|
||||
<a [title]="linkTitle()" class="badge" [style.background]="tag().color" [style.color]="tag().text_color">{{tag().name}}</a>
|
||||
}
|
||||
} @else if (loading) {
|
||||
<span class="placeholder-glow">
|
||||
|
||||
@@ -11,45 +11,3 @@ a, span {
|
||||
opacity: .5;
|
||||
font-style: italic;
|
||||
}
|
||||
|
||||
.split-parent,
|
||||
.split-child {
|
||||
overflow: hidden;
|
||||
white-space: nowrap;
|
||||
transition: color .2s ease;
|
||||
}
|
||||
|
||||
.split-parent {
|
||||
min-width: 1.5em;
|
||||
max-width: 10em;
|
||||
padding-inline: .4em;
|
||||
text-overflow: clip;
|
||||
|
||||
&:nth-last-child(2) { flex-shrink: 1e12; }
|
||||
&:nth-last-child(3) { flex-shrink: 1e18; }
|
||||
&:nth-last-child(4) { flex-shrink: 1e24; }
|
||||
&:nth-last-child(5) { flex-shrink: 1e30; }
|
||||
|
||||
&.collapsed {
|
||||
color: transparent !important;
|
||||
}
|
||||
}
|
||||
|
||||
.split-child {
|
||||
flex-shrink: 1e6;
|
||||
min-width: 2em;
|
||||
text-overflow: ellipsis;
|
||||
}
|
||||
|
||||
// Hovered segment gets priority, other parents can shrink to a sliver
|
||||
.split-pill {
|
||||
> .split-parent.focused,
|
||||
> .split-child.focused {
|
||||
flex-shrink: 1;
|
||||
text-overflow: ellipsis;
|
||||
}
|
||||
|
||||
&:has(> .focused) > .split-parent:not(.focused) {
|
||||
min-width: 0;
|
||||
}
|
||||
}
|
||||
@@ -6,7 +6,6 @@ import { of } from 'rxjs'
|
||||
import { Tag } from 'src/app/data/tag'
|
||||
import { PermissionsService } from 'src/app/services/permissions.service'
|
||||
import { TagService } from 'src/app/services/rest/tag.service'
|
||||
import { SettingsService } from 'src/app/services/settings.service'
|
||||
import { TagComponent } from './tag.component'
|
||||
|
||||
const tag: Tag = {
|
||||
@@ -70,88 +69,4 @@ describe('TagComponent', () => {
|
||||
expect(getCachedSpy).toHaveBeenCalledWith(1)
|
||||
expect(component.tag()).toEqual(tag)
|
||||
})
|
||||
describe('split badge', () => {
|
||||
const root: Tag = { id: 1, name: 'Root', color: '#000000' }
|
||||
const mid: Tag = { id: 2, name: 'Mid', color: '#111111', parent: 1 }
|
||||
const leaf: Tag = { id: 3, name: 'Leaf', color: '#222222', parent: 2 }
|
||||
|
||||
const segments = (): HTMLElement[] =>
|
||||
fixture.debugElement
|
||||
.queryAll(By.css('.split-pill > .badge'))
|
||||
.map((d) => d.nativeElement)
|
||||
|
||||
beforeEach(() => {
|
||||
jest.spyOn(permissionsService, 'currentUserCan').mockReturnValue(true)
|
||||
jest.spyOn(tagService, 'getCached').mockReturnValue(of(leaf))
|
||||
jest
|
||||
.spyOn(tagService, 'listAll')
|
||||
.mockReturnValue(
|
||||
of({ count: 3, all: [1, 2, 3], results: [root, mid, leaf] })
|
||||
)
|
||||
fixture.componentRef.setInput('tagID', 3)
|
||||
fixture.componentRef.setInput('split', true)
|
||||
fixture.detectChanges()
|
||||
})
|
||||
|
||||
it('should render ancestors as segments with the full path as title', () => {
|
||||
expect(segments().map((s) => s.textContent.trim())).toEqual([
|
||||
'Root',
|
||||
'Mid',
|
||||
'Leaf',
|
||||
])
|
||||
expect(
|
||||
fixture.debugElement.query(By.css('.split-pill')).nativeElement.title
|
||||
).toEqual('Root › Mid › Leaf')
|
||||
})
|
||||
|
||||
it('should render a plain badge when the setting is off', () => {
|
||||
jest.spyOn(TestBed.inject(SettingsService), 'get').mockReturnValue(false)
|
||||
fixture.componentRef.setInput('split', false)
|
||||
fixture.detectChanges()
|
||||
fixture.componentRef.setInput('split', true)
|
||||
fixture.detectChanges()
|
||||
expect(fixture.debugElement.query(By.css('.split-pill'))).toBeNull()
|
||||
expect(component.ancestorSegments).toEqual([])
|
||||
})
|
||||
|
||||
it('should emit the ancestor id when an ancestor segment is clicked', () => {
|
||||
const emitSpy = jest.spyOn(component.parentClick, 'emit')
|
||||
const event = new MouseEvent('click', { bubbles: true })
|
||||
const stopSpy = jest.spyOn(event, 'stopImmediatePropagation')
|
||||
segments()[1].dispatchEvent(event)
|
||||
expect(emitSpy).toHaveBeenCalledWith(2)
|
||||
expect(stopSpy).toHaveBeenCalled()
|
||||
})
|
||||
|
||||
it('should focus the hovered segment and animate width changes', () => {
|
||||
const [rootSeg, midSeg, leafSeg] = segments()
|
||||
const pill = rootSeg.parentElement
|
||||
// before / after widths per segment, as measured by setFocus
|
||||
const widths = new Map<HTMLElement, number[]>([
|
||||
[rootSeg, [20, 4]],
|
||||
[midSeg, [20, 60]],
|
||||
[leafSeg, [40, 40]],
|
||||
])
|
||||
const animate = jest.fn()
|
||||
for (const [seg, [before, after]] of widths) {
|
||||
seg.getAnimations = jest.fn(() => [])
|
||||
seg.animate = animate
|
||||
jest
|
||||
.spyOn(seg, 'getBoundingClientRect')
|
||||
.mockReturnValueOnce({ width: before } as DOMRect)
|
||||
.mockReturnValueOnce({ width: after } as DOMRect)
|
||||
}
|
||||
|
||||
midSeg.dispatchEvent(new MouseEvent('mouseenter'))
|
||||
expect(midSeg.classList).toContain('focused')
|
||||
expect(rootSeg.classList).not.toContain('focused')
|
||||
// only the segments whose width changed are animated
|
||||
expect(animate).toHaveBeenCalledTimes(2)
|
||||
|
||||
pill.dispatchEvent(new MouseEvent('mouseleave'))
|
||||
expect(segments().some((s) => s.classList.contains('focused'))).toBe(
|
||||
false
|
||||
)
|
||||
})
|
||||
})
|
||||
})
|
||||
@@ -1,27 +1,11 @@
|
||||
import {
|
||||
Component,
|
||||
computed,
|
||||
effect,
|
||||
inject,
|
||||
input,
|
||||
model,
|
||||
output,
|
||||
signal,
|
||||
} from '@angular/core'
|
||||
import { Component, effect, inject, input, model } from '@angular/core'
|
||||
import { Tag } from 'src/app/data/tag'
|
||||
import { SETTINGS_KEYS } from 'src/app/data/ui-settings'
|
||||
import {
|
||||
PermissionAction,
|
||||
PermissionsService,
|
||||
PermissionType,
|
||||
} from 'src/app/services/permissions.service'
|
||||
import { TagService } from 'src/app/services/rest/tag.service'
|
||||
import { SettingsService } from 'src/app/services/settings.service'
|
||||
import { getTagAncestors } from 'src/app/utils/collapse-tags'
|
||||
|
||||
const FOCUS_ANIMATION_MS = 200
|
||||
// matches the .split-parent min-width in tag.component.scss
|
||||
const COLLAPSE_BELOW_EM = 1.5
|
||||
|
||||
@Component({
|
||||
selector: 'pngx-tag',
|
||||
@@ -29,9 +13,8 @@ const COLLAPSE_BELOW_EM = 1.5
|
||||
styleUrls: ['./tag.component.scss'],
|
||||
})
|
||||
export class TagComponent {
|
||||
private readonly permissionsService = inject(PermissionsService)
|
||||
private readonly tagService = inject(TagService)
|
||||
private readonly settingsService = inject(SettingsService)
|
||||
private permissionsService = inject(PermissionsService)
|
||||
private tagService = inject(TagService)
|
||||
|
||||
readonly tag = model<Tag>(null)
|
||||
readonly tagID = input<number>(undefined)
|
||||
@@ -40,101 +23,27 @@ export class TagComponent {
|
||||
transform: (value: boolean | string) =>
|
||||
value === '' || value === true || value === 'true',
|
||||
})
|
||||
|
||||
readonly split = input(false)
|
||||
|
||||
readonly parentClick = output<number>()
|
||||
|
||||
private readonly ancestors = signal<Tag[]>([])
|
||||
|
||||
public get ancestorSegments(): Tag[] {
|
||||
return this.split() &&
|
||||
this.settingsService.get(SETTINGS_KEYS.TAGS_SHOW_HIERARCHY)
|
||||
? this.ancestors()
|
||||
: []
|
||||
}
|
||||
|
||||
public readonly path = computed(() =>
|
||||
[...this.ancestors(), this.tag()]
|
||||
.filter(Boolean)
|
||||
.map((t) => t.name)
|
||||
.join(' › ')
|
||||
)
|
||||
readonly showParents = input(false)
|
||||
|
||||
constructor() {
|
||||
effect(() => {
|
||||
const tagID = this.tagID()
|
||||
if (tagID && this.canViewTags) {
|
||||
this.tagService.getCached(tagID).subscribe((tag) => {
|
||||
this.tag.set(tag)
|
||||
})
|
||||
}
|
||||
})
|
||||
|
||||
// ancestors for split badges, from either tagID or a passed-in tag
|
||||
effect(() => {
|
||||
const id = this.tagID() || this.tag()?.id
|
||||
if (!this.split() || !id || !this.canViewTags) {
|
||||
this.ancestors.set([])
|
||||
return
|
||||
}
|
||||
this.tagService.listAll().subscribe((tags) => {
|
||||
this.ancestors.set(
|
||||
getTagAncestors(
|
||||
id,
|
||||
new Map<number, Tag>(tags.results.map((t) => [t.id, t]))
|
||||
if (tagID) {
|
||||
if (
|
||||
this.permissionsService.currentUserCan(
|
||||
PermissionAction.View,
|
||||
PermissionType.Tag
|
||||
)
|
||||
)
|
||||
})
|
||||
) {
|
||||
this.tagService.getCached(tagID).subscribe((tag) => {
|
||||
this.tag.set(tag)
|
||||
})
|
||||
}
|
||||
}
|
||||
})
|
||||
}
|
||||
|
||||
private get canViewTags(): boolean {
|
||||
return this.permissionsService.currentUserCan(
|
||||
PermissionAction.View,
|
||||
PermissionType.Tag
|
||||
)
|
||||
}
|
||||
|
||||
public get loading(): boolean {
|
||||
return this.tagService.loading
|
||||
}
|
||||
|
||||
/**
|
||||
* Gives the hovered segment priority (.focused class, see scss)
|
||||
* and animates each from old width to new
|
||||
*/
|
||||
setFocus(segment: HTMLElement, focused: HTMLElement | null) {
|
||||
const segments = Array.from(segment.children) as HTMLElement[]
|
||||
const before = segments.map((s) => s.getBoundingClientRect().width)
|
||||
for (const segment of segments) {
|
||||
segment.getAnimations().forEach((a) => a.cancel())
|
||||
segment.classList.toggle('focused', segment === focused)
|
||||
}
|
||||
const after = segments.map((s) => s.getBoundingClientRect().width)
|
||||
|
||||
segments.forEach((segment, i) => {
|
||||
const em = Number.parseFloat(getComputedStyle(segment).fontSize)
|
||||
segment.classList.toggle(
|
||||
'collapsed',
|
||||
!!focused &&
|
||||
segment.classList.contains('split-parent') &&
|
||||
after[i] < COLLAPSE_BELOW_EM * em
|
||||
)
|
||||
if (Math.abs(after[i] - before[i]) < 0.5) return
|
||||
segment.animate(
|
||||
[
|
||||
{ flex: `0 0 ${before[i]}px`, minWidth: 0 },
|
||||
{ flex: `0 0 ${after[i]}px`, minWidth: 0 },
|
||||
],
|
||||
{ duration: FOCUS_ANIMATION_MS, easing: 'ease' }
|
||||
)
|
||||
})
|
||||
}
|
||||
|
||||
onParentClick(event: MouseEvent, id: number) {
|
||||
event.preventDefault()
|
||||
event.stopImmediatePropagation()
|
||||
this.parentClick.emit(id)
|
||||
}
|
||||
}
|
||||
+2
-2
@@ -76,8 +76,8 @@
|
||||
}
|
||||
}
|
||||
@case (DisplayField.TAGS) {
|
||||
@for (tagID of doc.tags | collapseTags | async; track tagID) {
|
||||
<pngx-tag [tagID]="tagID" [split]="true" class="ms-1 fs-6" (click)="clickTag(tagID, $event)" (parentClick)="clickTag($event)" [clickable]="true" linkTitle="Filter by tag" i18n-title></pngx-tag>
|
||||
@for (tagID of doc.tags; track tagID) {
|
||||
<pngx-tag [tagID]="tagID" class="ms-1 fs-6" (click)="clickTag(tagID, $event)" [clickable]="true" linkTitle="Filter by tag" i18n-title></pngx-tag>
|
||||
}
|
||||
}
|
||||
@case (DisplayField.DOCUMENT_TYPE) {
|
||||
|
||||
-2
@@ -38,7 +38,6 @@ import {
|
||||
} from 'src/app/data/filter-rule-type'
|
||||
import { SavedView } from 'src/app/data/saved-view'
|
||||
import { IfPermissionsDirective } from 'src/app/directives/if-permissions.directive'
|
||||
import { CollapseTagsPipe } from 'src/app/pipes/collapse-tags.pipe'
|
||||
import { CorrespondentNamePipe } from 'src/app/pipes/correspondent-name.pipe'
|
||||
import { CustomDatePipe } from 'src/app/pipes/custom-date.pipe'
|
||||
import { DocumentTitlePipe } from 'src/app/pipes/document-title.pipe'
|
||||
@@ -76,7 +75,6 @@ import { WidgetFrameComponent } from '../widget-frame/widget-frame.component'
|
||||
DocumentTypeNamePipe,
|
||||
StoragePathNamePipe,
|
||||
AsyncPipe,
|
||||
CollapseTagsPipe,
|
||||
DocumentTitlePipe,
|
||||
CustomDatePipe,
|
||||
NgClass,
|
||||
|
||||
@@ -390,12 +390,14 @@ describe('DocumentDetailComponent', () => {
|
||||
expect(component.documentForm.get('title').value).toEqual('Foo Bar')
|
||||
})
|
||||
|
||||
it('should update the title form value without waiting for debounce or blur', () => {
|
||||
it('should update title before doc change if was not updated via debounce', () => {
|
||||
jest.useFakeTimers()
|
||||
initNormally()
|
||||
const input = component.titleInput.inputField.nativeElement
|
||||
input.value = 'Foo Bar'
|
||||
input.dispatchEvent(new Event('input'))
|
||||
component.titleInput.value = 'Foo Bar'
|
||||
component.titleInput.inputField.nativeElement.dispatchEvent(
|
||||
new Event('change')
|
||||
)
|
||||
jest.advanceTimersByTime(1000)
|
||||
expect(component.documentForm.get('title').value).toEqual('Foo Bar')
|
||||
})
|
||||
|
||||
|
||||
@@ -718,6 +718,7 @@ describe('BulkEditorComponent', () => {
|
||||
'Workflow will run on the selected documents in the background.'
|
||||
)
|
||||
expect(toast.mock.calls.at(-1)[0].actionName).toBeUndefined()
|
||||
expect(documentListViewService.selected.size).toBe(0)
|
||||
httpTestingController.match(
|
||||
`${environment.apiBaseUrl}documents/?page=1&page_size=50&ordering=-created&truncate_content=true&include_selection_data=true`
|
||||
) // list reload
|
||||
@@ -1292,7 +1293,7 @@ describe('BulkEditorComponent', () => {
|
||||
)
|
||||
req.flush({ result: 'OK', task_id: 'delete-2' })
|
||||
// ids aren't known when all matching documents are selected, so no undo
|
||||
expect(trackSpy).toHaveBeenLastCalledWith('delete-2', 2, null)
|
||||
expect(trackSpy).toHaveBeenLastCalledWith('delete-2', 0, null)
|
||||
httpTestingController.match(
|
||||
`${environment.apiBaseUrl}documents/?page=1&page_size=50&ordering=-created&truncate_content=true&include_selection_data=true`
|
||||
) // list reload
|
||||
@@ -1491,6 +1492,7 @@ describe('BulkEditorComponent', () => {
|
||||
httpTestingController.match(
|
||||
`${environment.apiBaseUrl}documents/?page=1&page_size=100000&fields=id`
|
||||
) // listAllFilteredIds
|
||||
expect(documentListViewService.selected.size).toEqual(0)
|
||||
|
||||
// Test with archiveFallback enabled
|
||||
modal.componentInstance.deleteOriginals.set(false)
|
||||
@@ -1512,6 +1514,7 @@ describe('BulkEditorComponent', () => {
|
||||
httpTestingController.match(
|
||||
`${environment.apiBaseUrl}documents/?page=1&page_size=100000&fields=id`
|
||||
) // listAllFilteredIds
|
||||
expect(documentListViewService.selected.size).toEqual(0)
|
||||
})
|
||||
|
||||
it('should support merging documents as versions', () => {
|
||||
@@ -1559,6 +1562,7 @@ describe('BulkEditorComponent', () => {
|
||||
httpTestingController.match(
|
||||
`${environment.apiBaseUrl}documents/?page=1&page_size=100000&fields=id`
|
||||
)
|
||||
expect(documentListViewService.selected.size).toEqual(0)
|
||||
jest.advanceTimersByTime(400)
|
||||
expect(toastSpy).toHaveBeenCalledWith(
|
||||
expect.objectContaining({
|
||||
@@ -1713,6 +1717,7 @@ describe('BulkEditorComponent', () => {
|
||||
expect(notification.actionName).toBeUndefined()
|
||||
}
|
||||
expect(reload).not.toHaveBeenCalled()
|
||||
expect(documentListViewService.selected.size).toBe(0)
|
||||
expect(documentListViewService.allSelected).toBe(false)
|
||||
TestBed.inject(TaskNotificationService).complete({
|
||||
task_id: 'queued-operation',
|
||||
|
||||
@@ -329,6 +329,7 @@ export class BulkEditorComponent
|
||||
.subscribe({
|
||||
next: () => {
|
||||
modal.close()
|
||||
this.list.selectNone()
|
||||
this.toastService.showInfo(
|
||||
$localize`Workflow will run on the selected documents in the background.`
|
||||
)
|
||||
@@ -365,6 +366,7 @@ export class BulkEditorComponent
|
||||
) {
|
||||
const bulk = this.getSelectionSize() > 1
|
||||
modal?.close()
|
||||
this.list.selectNone()
|
||||
if (onQueued) {
|
||||
onQueued(taskId)
|
||||
} else {
|
||||
|
||||
+2
-2
@@ -36,8 +36,8 @@
|
||||
{{document().title | documentTitle}}
|
||||
}
|
||||
@if (displayFields().includes(DisplayField.TAGS)) {
|
||||
@for (tagID of document().tags | collapseTags | async; track tagID) {
|
||||
<pngx-tag [tagID]="tagID" [split]="true" linkTitle="Filter by tag" i18n-linkTitle class="ms-1" (click)="clickTag.emit(tagID);$event.stopPropagation()" (parentClick)="clickTag.emit($event)" [clickable]="clickTag.observers.length"></pngx-tag>
|
||||
@for (tagID of document().tags; track tagID) {
|
||||
<pngx-tag [tagID]="tagID" linkTitle="Filter by tag" i18n-linkTitle class="ms-1" (click)="clickTag.emit(tagID);$event.stopPropagation()" [clickable]="clickTag.observers.length"></pngx-tag>
|
||||
}
|
||||
}
|
||||
} @else {
|
||||
|
||||
-2
@@ -20,7 +20,6 @@ import {
|
||||
Document,
|
||||
} from 'src/app/data/document'
|
||||
import { SETTINGS_KEYS } from 'src/app/data/ui-settings'
|
||||
import { CollapseTagsPipe } from 'src/app/pipes/collapse-tags.pipe'
|
||||
import { CorrespondentNamePipe } from 'src/app/pipes/correspondent-name.pipe'
|
||||
import { CustomDatePipe } from 'src/app/pipes/custom-date.pipe'
|
||||
import { DocumentTitlePipe } from 'src/app/pipes/document-title.pipe'
|
||||
@@ -46,7 +45,6 @@ import { LoadingComponentWithPermissions } from '../../loading-component/loading
|
||||
TagComponent,
|
||||
CustomFieldDisplayComponent,
|
||||
AsyncPipe,
|
||||
CollapseTagsPipe,
|
||||
NgOptimizedImage,
|
||||
UsernamePipe,
|
||||
CorrespondentNamePipe,
|
||||
|
||||
+4
-5
@@ -19,12 +19,11 @@
|
||||
}
|
||||
|
||||
@if (document() && displayFields().includes(DisplayField.TAGS)) {
|
||||
@let visibleTagIDs = document().tags | collapseTags | async;
|
||||
<div class="tags d-flex flex-column text-end position-absolute me-1 fs-6" [class.tags-no-wrap]="visibleTagIDs?.length > 3">
|
||||
@for (tagID of getTagIDs(visibleTagIDs); track tagID) {
|
||||
<pngx-tag [tagID]="tagID" [split]="true" (click)="clickTag.emit(tagID);$event.stopPropagation()" (parentClick)="clickTag.emit($event)" [clickable]="true" linkTitle="Toggle tag filter" i18n-linkTitle></pngx-tag>
|
||||
<div class="tags d-flex flex-column text-end position-absolute me-1 fs-6" [class.tags-no-wrap]="document().tags.length > 3">
|
||||
@for (tagID of tagIDs; track tagID) {
|
||||
<pngx-tag [tagID]="tagID" (click)="clickTag.emit(tagID);$event.stopPropagation()" [clickable]="true" linkTitle="Toggle tag filter" i18n-linkTitle></pngx-tag>
|
||||
}
|
||||
@if (getMoreTags(visibleTagIDs); as moreTags) {
|
||||
@if (moreTags) {
|
||||
<div>
|
||||
<span class="badge text-dark">+ {{moreTags}}</span>
|
||||
</div>
|
||||
|
||||
+1
-2
@@ -103,8 +103,7 @@ a {
|
||||
line-height: 1;
|
||||
|
||||
&.tags-no-wrap {
|
||||
// split-pill parents clip to a stub instead (see TagComponent)
|
||||
::ng-deep .badge:not(.split-parent) {
|
||||
::ng-deep .badge {
|
||||
display: inline-block;
|
||||
max-width: 100%;
|
||||
white-space: nowrap;
|
||||
|
||||
+2
-33
@@ -4,9 +4,6 @@ import { provideHttpClientTesting } from '@angular/common/http/testing'
|
||||
import { ComponentFixture, TestBed } from '@angular/core/testing'
|
||||
import { By } from '@angular/platform-browser'
|
||||
import { RouterTestingModule } from '@angular/router/testing'
|
||||
import { of } from 'rxjs'
|
||||
import { PermissionsService } from 'src/app/services/permissions.service'
|
||||
import { TagService } from 'src/app/services/rest/tag.service'
|
||||
import { TagComponent } from '../../common/tag/tag.component'
|
||||
import { DocumentCardSmallComponent } from './document-card-small.component'
|
||||
|
||||
@@ -100,36 +97,8 @@ describe('DocumentCardSmallComponent', () => {
|
||||
).toHaveLength(6)
|
||||
})
|
||||
|
||||
it('should hide parent tags implied by a child and count pills for the hidden tag counter', () => {
|
||||
jest
|
||||
.spyOn(TestBed.inject(PermissionsService), 'currentUserCan')
|
||||
.mockReturnValue(true)
|
||||
jest.spyOn(TestBed.inject(TagService), 'listAll').mockReturnValue(
|
||||
of({
|
||||
count: 8,
|
||||
all: [1, 2, 3, 4, 5, 6, 7, 8],
|
||||
results: [
|
||||
{ id: 1, name: 'Parent' },
|
||||
{ id: 2, name: 'Child', parent: 1 },
|
||||
...[3, 4, 5, 6, 7, 8].map((id) => ({ id, name: `Tag ${id}` })),
|
||||
],
|
||||
})
|
||||
)
|
||||
fixture.componentRef.setInput('document', {
|
||||
...component.document(),
|
||||
tags: [1, 2, 3, 4, 5, 6, 7, 8],
|
||||
})
|
||||
fixture.detectChanges()
|
||||
|
||||
const tags = fixture.debugElement.queryAll(By.directive(TagComponent))
|
||||
expect(tags.map((t) => t.componentInstance.tagID())).toEqual([
|
||||
2, 3, 4, 5, 6,
|
||||
])
|
||||
expect(fixture.nativeElement.textContent).toContain('+ 2')
|
||||
})
|
||||
|
||||
it('should clear hidden tag counter when tag count falls below the limit', () => {
|
||||
expect(component.getMoreTags(component.document().tags)).toEqual(3)
|
||||
expect(component.moreTags).toEqual(3)
|
||||
|
||||
fixture.componentRef.setInput('document', {
|
||||
...component.document(),
|
||||
@@ -137,7 +106,7 @@ describe('DocumentCardSmallComponent', () => {
|
||||
})
|
||||
fixture.detectChanges()
|
||||
|
||||
expect(component.getMoreTags(component.document().tags)).toBeNull()
|
||||
expect(component.moreTags).toBeNull()
|
||||
expect(fixture.nativeElement.textContent).not.toContain('+ 3')
|
||||
})
|
||||
|
||||
|
||||
+14
-12
@@ -20,7 +20,6 @@ import {
|
||||
Document,
|
||||
} from 'src/app/data/document'
|
||||
import { SETTINGS_KEYS } from 'src/app/data/ui-settings'
|
||||
import { CollapseTagsPipe } from 'src/app/pipes/collapse-tags.pipe'
|
||||
import { CorrespondentNamePipe } from 'src/app/pipes/correspondent-name.pipe'
|
||||
import { CustomDatePipe } from 'src/app/pipes/custom-date.pipe'
|
||||
import { DocumentTitlePipe } from 'src/app/pipes/document-title.pipe'
|
||||
@@ -46,7 +45,6 @@ import { LoadingComponentWithPermissions } from '../../loading-component/loading
|
||||
TagComponent,
|
||||
CustomFieldDisplayComponent,
|
||||
AsyncPipe,
|
||||
CollapseTagsPipe,
|
||||
NgOptimizedImage,
|
||||
UsernamePipe,
|
||||
CorrespondentNamePipe,
|
||||
@@ -92,13 +90,12 @@ export class DocumentCardSmallComponent
|
||||
@Output()
|
||||
clickStoragePath = new EventEmitter<number>()
|
||||
|
||||
private get tagLimit(): number {
|
||||
return this.document()?.notes.length > 0 ? 6 : 7
|
||||
}
|
||||
|
||||
getMoreTags(tagIDs: number[]): number {
|
||||
const limit = this.tagLimit
|
||||
return tagIDs?.length > limit ? tagIDs.length - (limit - 1) : null
|
||||
get moreTags(): number {
|
||||
const document = this.document()
|
||||
const limit = document?.notes.length > 0 ? 6 : 7
|
||||
return document?.tags.length > limit
|
||||
? document.tags.length - (limit - 1)
|
||||
: null
|
||||
}
|
||||
|
||||
@ViewChild('popupPreview') popupPreview: PreviewPopupComponent
|
||||
@@ -119,9 +116,14 @@ export class DocumentCardSmallComponent
|
||||
return this.documentService.getDownloadUrl(this.document().id)
|
||||
}
|
||||
|
||||
getTagIDs(tagIDs: number[]): number[] {
|
||||
const limit = this.tagLimit
|
||||
return tagIDs?.length > limit ? tagIDs.slice(0, limit - 1) : tagIDs
|
||||
get tagIDs() {
|
||||
const document = this.document()
|
||||
const limit = document.notes.length > 0 ? 6 : 7
|
||||
if (document.tags.length > limit) {
|
||||
return document.tags.slice(0, limit - 1)
|
||||
} else {
|
||||
return document.tags
|
||||
}
|
||||
}
|
||||
|
||||
mouseLeaveCard() {
|
||||
|
||||
@@ -335,8 +335,8 @@
|
||||
<a routerLink="/documents/{{d.id}}" title="Edit document" i18n-title style="overflow-wrap: anywhere;">{{d.title | documentTitle}}</a>
|
||||
}
|
||||
@if (activeDisplayFields.includes(DisplayField.TAGS)) {
|
||||
@for (tagID of d.tags | collapseTags | async; track tagID) {
|
||||
<pngx-tag [tagID]="tagID" [split]="true" class="ms-1 fs-6" clickable="true" linkTitle="Filter by tag" i18n-linkTitle (click)="clickTag(tagID);$event.stopPropagation()" (parentClick)="clickTag($event)"></pngx-tag>
|
||||
@for (tagID of d.tags; track tagID) {
|
||||
<pngx-tag [tagID]="tagID" class="ms-1 fs-6" clickable="true" linkTitle="Filter by tag" i18n-linkTitle (click)="clickTag(tagID);$event.stopPropagation()"></pngx-tag>
|
||||
}
|
||||
}
|
||||
</td>
|
||||
|
||||
@@ -39,7 +39,6 @@ import {
|
||||
SortableDirective,
|
||||
SortEvent,
|
||||
} from 'src/app/directives/sortable.directive'
|
||||
import { CollapseTagsPipe } from 'src/app/pipes/collapse-tags.pipe'
|
||||
import { CorrespondentNamePipe } from 'src/app/pipes/correspondent-name.pipe'
|
||||
import { CustomDatePipe } from 'src/app/pipes/custom-date.pipe'
|
||||
import { DocumentTitlePipe } from 'src/app/pipes/document-title.pipe'
|
||||
@@ -97,7 +96,6 @@ import { SaveViewConfigDialogComponent } from './save-view-config-dialog/save-vi
|
||||
StoragePathNamePipe,
|
||||
BootstrapIconComponent,
|
||||
AsyncPipe,
|
||||
CollapseTagsPipe,
|
||||
FormsModule,
|
||||
ReactiveFormsModule,
|
||||
NgTemplateOutlet,
|
||||
|
||||
@@ -94,7 +94,6 @@ export const SETTINGS_KEYS = {
|
||||
DEFAULT_PERMS_VIEW_GROUPS: 'general-settings:permissions:default-view-groups',
|
||||
DEFAULT_PERMS_EDIT_USERS: 'general-settings:permissions:default-edit-users',
|
||||
DEFAULT_PERMS_EDIT_GROUPS: 'general-settings:permissions:default-edit-groups',
|
||||
TAGS_SHOW_HIERARCHY: 'general-settings:tags:show-hierarchy',
|
||||
DOCUMENT_EDITING_REMOVE_INBOX_TAGS:
|
||||
'general-settings:document-editing:remove-inbox-tags',
|
||||
DOCUMENT_EDITING_OVERLAY_THUMBNAIL:
|
||||
@@ -306,11 +305,6 @@ export const SETTINGS: UiSetting[] = [
|
||||
type: 'string',
|
||||
default: '',
|
||||
},
|
||||
{
|
||||
key: SETTINGS_KEYS.TAGS_SHOW_HIERARCHY,
|
||||
type: 'boolean',
|
||||
default: true,
|
||||
},
|
||||
{
|
||||
key: SETTINGS_KEYS.DOCUMENT_EDITING_REMOVE_INBOX_TAGS,
|
||||
type: 'boolean',
|
||||
|
||||
@@ -1,67 +0,0 @@
|
||||
import { provideHttpClient, withInterceptorsFromDi } from '@angular/common/http'
|
||||
import { provideHttpClientTesting } from '@angular/common/http/testing'
|
||||
import { TestBed } from '@angular/core/testing'
|
||||
import { firstValueFrom, of, throwError } from 'rxjs'
|
||||
import { Tag } from '../data/tag'
|
||||
import { PermissionsService } from '../services/permissions.service'
|
||||
import { TagService } from '../services/rest/tag.service'
|
||||
import { SettingsService } from '../services/settings.service'
|
||||
import { CollapseTagsPipe } from './collapse-tags.pipe'
|
||||
|
||||
const tags: Tag[] = [
|
||||
{ id: 1, name: 'Parent' },
|
||||
{ id: 2, name: 'Child', parent: 1 },
|
||||
{ id: 3, name: 'Other' },
|
||||
]
|
||||
|
||||
describe('CollapseTagsPipe', () => {
|
||||
let pipe: CollapseTagsPipe
|
||||
let tagService: TagService
|
||||
let settingsService: SettingsService
|
||||
let permissionsService: PermissionsService
|
||||
|
||||
beforeEach(() => {
|
||||
TestBed.configureTestingModule({
|
||||
providers: [
|
||||
CollapseTagsPipe,
|
||||
provideHttpClient(withInterceptorsFromDi()),
|
||||
provideHttpClientTesting(),
|
||||
],
|
||||
})
|
||||
pipe = TestBed.inject(CollapseTagsPipe)
|
||||
tagService = TestBed.inject(TagService)
|
||||
settingsService = TestBed.inject(SettingsService)
|
||||
permissionsService = TestBed.inject(PermissionsService)
|
||||
jest.spyOn(permissionsService, 'currentUserCan').mockReturnValue(true)
|
||||
jest.spyOn(settingsService, 'get').mockReturnValue(true)
|
||||
jest
|
||||
.spyOn(tagService, 'listAll')
|
||||
.mockReturnValue(of({ count: 3, all: [1, 2, 3], results: tags }))
|
||||
})
|
||||
|
||||
it('should drop parents implied by a selected child', async () => {
|
||||
expect(await firstValueFrom(pipe.transform([1, 2, 3]))).toEqual([2, 3])
|
||||
})
|
||||
|
||||
it('should pass ids through when hierarchy display is off or not permitted', async () => {
|
||||
jest.spyOn(settingsService, 'get').mockReturnValue(false)
|
||||
expect(await firstValueFrom(pipe.transform([1, 2]))).toEqual([1, 2])
|
||||
|
||||
jest.spyOn(settingsService, 'get').mockReturnValue(true)
|
||||
jest.spyOn(permissionsService, 'currentUserCan').mockReturnValue(false)
|
||||
expect(await firstValueFrom(pipe.transform([1, 2]))).toEqual([1, 2])
|
||||
expect(tagService.listAll).not.toHaveBeenCalled()
|
||||
})
|
||||
|
||||
it('should handle empty input', async () => {
|
||||
expect(await firstValueFrom(pipe.transform(null))).toEqual([])
|
||||
expect(await firstValueFrom(pipe.transform([]))).toEqual([])
|
||||
})
|
||||
|
||||
it('should fall back to the original ids if tags fail to load', async () => {
|
||||
jest
|
||||
.spyOn(tagService, 'listAll')
|
||||
.mockReturnValue(throwError(() => new Error('failed')))
|
||||
expect(await firstValueFrom(pipe.transform([1, 2]))).toEqual([1, 2])
|
||||
})
|
||||
})
|
||||
@@ -1,47 +0,0 @@
|
||||
import { inject, Pipe, PipeTransform } from '@angular/core'
|
||||
import { catchError, map, Observable, of } from 'rxjs'
|
||||
import { Tag } from '../data/tag'
|
||||
import { SETTINGS_KEYS } from '../data/ui-settings'
|
||||
import {
|
||||
PermissionAction,
|
||||
PermissionsService,
|
||||
PermissionType,
|
||||
} from '../services/permissions.service'
|
||||
import { TagService } from '../services/rest/tag.service'
|
||||
import { SettingsService } from '../services/settings.service'
|
||||
import { collapseImpliedAncestors } from '../utils/collapse-tags'
|
||||
|
||||
/**
|
||||
* Drops tag IDs whose descendant is also in the list for when the child's
|
||||
* Tag badge already shows its parent.
|
||||
*/
|
||||
@Pipe({
|
||||
name: 'collapseTags',
|
||||
})
|
||||
export class CollapseTagsPipe implements PipeTransform {
|
||||
private readonly permissionsService = inject(PermissionsService)
|
||||
private readonly tagService = inject(TagService)
|
||||
private readonly settingsService = inject(SettingsService)
|
||||
|
||||
transform(ids: number[]): Observable<number[]> {
|
||||
if (
|
||||
!ids?.length ||
|
||||
!this.settingsService.get(SETTINGS_KEYS.TAGS_SHOW_HIERARCHY) ||
|
||||
!this.permissionsService.currentUserCan(
|
||||
PermissionAction.View,
|
||||
PermissionType.Tag
|
||||
)
|
||||
) {
|
||||
return of(ids ?? [])
|
||||
}
|
||||
return this.tagService.listAll().pipe(
|
||||
map((tags) =>
|
||||
collapseImpliedAncestors(
|
||||
ids,
|
||||
new Map<number, Tag>(tags.results.map((t) => [t.id, t]))
|
||||
)
|
||||
),
|
||||
catchError(() => of(ids))
|
||||
)
|
||||
}
|
||||
}
|
||||
@@ -1,47 +0,0 @@
|
||||
import { Tag } from '../data/tag'
|
||||
import { collapseImpliedAncestors, getTagAncestors } from './collapse-tags'
|
||||
|
||||
const tags: Tag[] = [
|
||||
{ id: 1, name: 'Finance' },
|
||||
{ id: 2, name: 'Account', parent: 1 },
|
||||
{ id: 3, name: 'Mortgage', parent: 2 },
|
||||
{ id: 4, name: 'Taxes', parent: 1 },
|
||||
{ id: 5, name: 'Inbox' },
|
||||
]
|
||||
const tagsById = new Map(tags.map((t) => [t.id, t]))
|
||||
|
||||
describe('collapseImpliedAncestors', () => {
|
||||
it('drops ancestors that are implied by a descendant in the list', () => {
|
||||
expect(collapseImpliedAncestors([1, 2, 3, 4, 5], tagsById)).toEqual([
|
||||
3, 4, 5,
|
||||
])
|
||||
})
|
||||
|
||||
it('keeps a parent with none of its children present', () => {
|
||||
expect(collapseImpliedAncestors([1, 5], tagsById)).toEqual([1, 5])
|
||||
})
|
||||
|
||||
it('keeps unknown (private) ids and handles empty input', () => {
|
||||
expect(collapseImpliedAncestors([99, 2], tagsById)).toEqual([99, 2])
|
||||
expect(collapseImpliedAncestors([], tagsById)).toEqual([])
|
||||
expect(collapseImpliedAncestors(null, tagsById)).toEqual([])
|
||||
})
|
||||
})
|
||||
|
||||
describe('getTagAncestors', () => {
|
||||
it('returns ancestors root first', () => {
|
||||
expect(getTagAncestors(3, tagsById).map((t) => t.id)).toEqual([1, 2])
|
||||
expect(getTagAncestors(5, tagsById)).toEqual([])
|
||||
})
|
||||
|
||||
it('stops at a parent that is not visible or forms a cycle', () => {
|
||||
const hidden = new Map<number, Tag>([[6, { id: 6, name: 'x', parent: 99 }]])
|
||||
expect(getTagAncestors(6, hidden)).toEqual([])
|
||||
|
||||
const cyclic = new Map<number, Tag>([
|
||||
[7, { id: 7, name: 'a', parent: 8 }],
|
||||
[8, { id: 8, name: 'b', parent: 7 }],
|
||||
])
|
||||
expect(getTagAncestors(7, cyclic).map((t) => t.id)).toEqual([7, 8])
|
||||
})
|
||||
})
|
||||
@@ -1,43 +0,0 @@
|
||||
import { Tag } from '../data/tag'
|
||||
|
||||
/**
|
||||
* Collapses implied ancestor tags from a list of tag IDs such that if a tag's
|
||||
* ancestor is also present in the list, the ancestor is removed. For example
|
||||
* [Finance, Account, Mortgage] -> [Mortgage]
|
||||
*/
|
||||
export function collapseImpliedAncestors(
|
||||
ids: number[],
|
||||
tagsById: Map<number, Tag>
|
||||
): number[] {
|
||||
if (!ids?.length) return ids ?? []
|
||||
const present = new Set(ids)
|
||||
const implied = new Set<number>()
|
||||
for (const id of ids) {
|
||||
const seen = new Set<number>()
|
||||
let parentId = tagsById.get(id)?.parent
|
||||
while (parentId && !seen.has(parentId)) {
|
||||
seen.add(parentId)
|
||||
if (present.has(parentId)) implied.add(parentId)
|
||||
parentId = tagsById.get(parentId)?.parent
|
||||
}
|
||||
}
|
||||
return ids.filter((id) => !implied.has(id))
|
||||
}
|
||||
|
||||
/**
|
||||
* Returns an array of ancestor tags for the given tag ID, starting from the
|
||||
* root and ending with the immediate parent.
|
||||
*/
|
||||
export function getTagAncestors(id: number, tagsById: Map<number, Tag>): Tag[] {
|
||||
const chain: Tag[] = []
|
||||
const seen = new Set<number>()
|
||||
let parentId = tagsById.get(id)?.parent
|
||||
while (parentId && !seen.has(parentId)) {
|
||||
seen.add(parentId)
|
||||
const parent = tagsById.get(parentId)
|
||||
if (!parent) break
|
||||
chain.unshift(parent)
|
||||
parentId = parent.parent
|
||||
}
|
||||
return chain
|
||||
}
|
||||
@@ -8,7 +8,7 @@ export const environment = {
|
||||
apiVersion: '10', // match src/paperless/settings.py
|
||||
appTitle: DEFAULT_APP_TITLE,
|
||||
tag: 'prod',
|
||||
version: '3.5.1',
|
||||
version: '3.4.2',
|
||||
webSocketHost: window.location.host,
|
||||
webSocketProtocol: window.location.protocol == 'https:' ? 'wss:' : 'ws:',
|
||||
webSocketBaseUrl: base_url.pathname + 'ws/',
|
||||
|
||||
File diff suppressed because it is too large.
Load diff
File diff suppressed because it is too large.
Load diff
File diff suppressed because it is too large.
Load diff
File diff suppressed because it is too large.
Load diff
File diff suppressed because it is too large.
Load diff
File diff suppressed because it is too large.
Load diff
File diff suppressed because it is too large.
Load diff
File diff suppressed because it is too large.
Load diff
File diff suppressed because it is too large.
Load diff
File diff suppressed because it is too large.
Load diff
File diff suppressed because it is too large.
Load diff
File diff suppressed because it is too large.
Load diff
File diff suppressed because it is too large.
Load diff
File diff suppressed because it is too large.
Load diff
File diff suppressed because it is too large.
Load diff
File diff suppressed because it is too large.
Load diff
File diff suppressed because it is too large.
Load diff
File diff suppressed because it is too large.
Load diff
File diff suppressed because it is too large.
Load diff
File diff suppressed because it is too large.
Load diff
File diff suppressed because it is too large.
Load diff
File diff suppressed because it is too large.
Load diff
File diff suppressed because it is too large.
Load diff
File diff suppressed because it is too large.
Load diff
File diff suppressed because it is too large.
Load diff
File diff suppressed because it is too large.
Load diff
File diff suppressed because it is too large.
Load diff
File diff suppressed because it is too large.
Load diff
File diff suppressed because it is too large.
Load diff
File diff suppressed because it is too large.
Load diff
File diff suppressed because it is too large.
Load diff
File diff suppressed because it is too large.
Load diff
File diff suppressed because it is too large.
Load diff
File diff suppressed because it is too large.
Load diff
File diff suppressed because it is too large.
Load diff
File diff suppressed because it is too large.
Load diff
File diff suppressed because it is too large.
Load diff
File diff suppressed because it is too large.
Load diff
File diff suppressed because it is too large.
Load diff
File diff suppressed because it is too large.
Load diff
File diff suppressed because it is too large.
Load diff
File diff suppressed because it is too large.
Load diff
File diff suppressed because it is too large.
Load diff
File diff suppressed because it is too large.
Load diff
File diff suppressed because it is too large.
Load diff
File diff suppressed because it is too large.
Load diff
File diff suppressed because it is too large.
Load diff
+126
-69
@@ -39,6 +39,8 @@ from documents.models import Tag
|
||||
from documents.permissions import PermissionsSpec
|
||||
from documents.permissions import set_permissions_for_objects
|
||||
from documents.plugins.helpers import DocumentsStatusManager
|
||||
from documents.task_tracking import queue_consumption
|
||||
from documents.task_tracking import queue_processing_tasks
|
||||
from documents.tasks import bulk_update_documents
|
||||
from documents.tasks import consume_file
|
||||
from documents.tasks import remove_document_from_index
|
||||
@@ -432,7 +434,12 @@ def delete(doc_ids: list[int]) -> Literal["OK"]:
|
||||
return "OK"
|
||||
|
||||
|
||||
def reprocess(doc_ids: list[int], *, remote_ocr: bool = False) -> Literal["OK"]:
|
||||
def reprocess(
|
||||
doc_ids: list[int],
|
||||
*,
|
||||
remote_ocr: bool = False,
|
||||
parent_task_id: str | None = None,
|
||||
) -> Literal["OK"]:
|
||||
"""
|
||||
Re-run parsing for the given documents.
|
||||
|
||||
@@ -457,6 +464,16 @@ def reprocess(doc_ids: list[int], *, remote_ocr: bool = False) -> Literal["OK"]:
|
||||
.values_list("source_id", flat=True)
|
||||
.distinct()
|
||||
)
|
||||
if parent_task_id is not None:
|
||||
from documents.task_tracking import queue_reprocessing_tasks
|
||||
|
||||
queue_reprocessing_tasks(
|
||||
parent_task_id,
|
||||
list(source_ids),
|
||||
remote_ocr=remote_ocr,
|
||||
)
|
||||
return "OK"
|
||||
|
||||
for source_id in source_ids:
|
||||
update_document_content_maybe_archive_file.apply_async(
|
||||
kwargs={"document_id": source_id, "remote_ocr": remote_ocr},
|
||||
@@ -521,6 +538,7 @@ def rotate(
|
||||
source_mode: SourceMode = SourceModeChoices.LATEST_VERSION,
|
||||
user: User | None = None,
|
||||
trigger_source: PaperlessTask.TriggerSource = PaperlessTask.TriggerSource.WEB_UI,
|
||||
parent_task_id: str | None = None,
|
||||
) -> Literal["OK"]:
|
||||
logger.info(
|
||||
f"Attempting to rotate {len(doc_ids)} documents by {degrees} degrees.",
|
||||
@@ -539,7 +557,6 @@ def rotate(
|
||||
pair = _resolve_root_and_source_doc(doc, source_mode=source_mode)
|
||||
docs_by_root_id.setdefault(pair.root_doc.id, pair)
|
||||
|
||||
failed_ids = []
|
||||
for pair in docs_by_root_id.values():
|
||||
if pair.source_doc.mime_type != "application/pdf":
|
||||
logger.warning(
|
||||
@@ -559,28 +576,22 @@ def rotate(
|
||||
if user is not None:
|
||||
overrides.actor_id = user.id
|
||||
|
||||
consume_file.apply_async(
|
||||
kwargs={
|
||||
"input_doc": ConsumableDocument(
|
||||
source=DocumentSource.ConsumeFolder,
|
||||
original_file=filepath,
|
||||
root_document_id=pair.root_doc.id,
|
||||
),
|
||||
"overrides": overrides,
|
||||
},
|
||||
headers={"trigger_source": trigger_source},
|
||||
queue_consumption(
|
||||
input_doc=ConsumableDocument(
|
||||
source=DocumentSource.ConsumeFolder,
|
||||
original_file=filepath,
|
||||
root_document_id=pair.root_doc.id,
|
||||
),
|
||||
overrides=overrides,
|
||||
trigger_source=trigger_source,
|
||||
parent_task_id=parent_task_id,
|
||||
)
|
||||
logger.info(
|
||||
f"Queued new rotated version for document {pair.root_doc.id} by {degrees} degrees",
|
||||
)
|
||||
except Exception as e:
|
||||
failed_ids.append(pair.root_doc.id)
|
||||
logger.exception(f"Error rotating document {pair.root_doc.id}: {e}")
|
||||
|
||||
if failed_ids:
|
||||
raise RuntimeError(
|
||||
f"Rotation failed for document(s): {failed_ids}. Check logs for details.",
|
||||
)
|
||||
raise
|
||||
|
||||
return "OK"
|
||||
|
||||
@@ -594,13 +605,15 @@ def merge(
|
||||
source_mode: SourceMode = SourceModeChoices.LATEST_VERSION,
|
||||
user: User | None = None,
|
||||
trigger_source: PaperlessTask.TriggerSource = PaperlessTask.TriggerSource.WEB_UI,
|
||||
) -> Literal["OK"]:
|
||||
parent_task_id: str | None = None,
|
||||
) -> Literal["OK"] | dict:
|
||||
logger.info(
|
||||
f"Attempting to merge {len(doc_ids)} documents into a single document.",
|
||||
)
|
||||
qs = Document.objects.select_related("root_document").filter(id__in=doc_ids)
|
||||
docs_by_id = {doc.id: doc for doc in qs}
|
||||
affected_docs: list[int] = []
|
||||
failed_ids: list[int] = []
|
||||
handoff_asn: int | None = None
|
||||
with pdf_ops.PdfMerger() as merger:
|
||||
# use doc_ids to preserve order
|
||||
@@ -626,11 +639,14 @@ def merge(
|
||||
if handoff_asn is None and doc.archive_serial_number is not None:
|
||||
handoff_asn = doc.archive_serial_number
|
||||
except Exception as e:
|
||||
failed_ids.append(doc.id)
|
||||
logger.exception(
|
||||
f"Error merging document {doc.id}, it will not be included in the merge: {e}",
|
||||
)
|
||||
if len(affected_docs) == 0:
|
||||
logger.warning("No documents were merged")
|
||||
if parent_task_id is not None:
|
||||
raise ValueError("None of the selected documents could be merged.")
|
||||
return "OK"
|
||||
|
||||
filepath = (
|
||||
@@ -673,6 +689,23 @@ def merge(
|
||||
overrides=overrides,
|
||||
).set(headers={"trigger_source": trigger_source})
|
||||
|
||||
if parent_task_id is not None:
|
||||
backup = (
|
||||
release_archive_serial_numbers(affected_docs) if delete_originals else {}
|
||||
)
|
||||
queue_processing_tasks(
|
||||
parent_task_id,
|
||||
[consume_task],
|
||||
delete_ids=affected_docs if delete_originals else None,
|
||||
asn_backup=backup,
|
||||
)
|
||||
if failed_ids:
|
||||
return {
|
||||
"failed_document_ids": failed_ids,
|
||||
"error_message": f"Documents {failed_ids} were excluded from the merge.",
|
||||
}
|
||||
return "OK"
|
||||
|
||||
if delete_originals:
|
||||
backup = release_archive_serial_numbers(affected_docs)
|
||||
logger.info(
|
||||
@@ -692,22 +725,6 @@ def merge(
|
||||
return "OK"
|
||||
|
||||
|
||||
def validate_merge_as_versions_state(
|
||||
documents: list[Document],
|
||||
root_document_id: int,
|
||||
) -> None:
|
||||
if any(document.root_document_id is not None for document in documents):
|
||||
raise ValueError("Only top-level documents can be merged as versions.")
|
||||
|
||||
source_ids = [
|
||||
document.pk for document in documents if document.pk != root_document_id
|
||||
]
|
||||
if Document.global_objects.filter(root_document_id__in=source_ids).exists():
|
||||
raise ValueError(
|
||||
"Documents with existing versions cannot be merged into another document.",
|
||||
)
|
||||
|
||||
|
||||
def merge_as_versions(
|
||||
doc_ids: list[int],
|
||||
*,
|
||||
@@ -725,8 +742,15 @@ def merge_as_versions(
|
||||
)
|
||||
documents_by_id = {document.id: document for document in documents}
|
||||
|
||||
validate_merge_as_versions_state(documents, root_document_id)
|
||||
if any(document.root_document_id is not None for document in documents):
|
||||
raise ValueError("Only top-level documents can be merged as versions.")
|
||||
|
||||
source_ids = [doc_id for doc_id in doc_ids if doc_id != root_document_id]
|
||||
if Document.global_objects.filter(root_document_id__in=source_ids).exists():
|
||||
raise ValueError(
|
||||
"Documents with existing versions cannot be merged into another document.",
|
||||
)
|
||||
|
||||
root_document = documents_by_id[root_document_id]
|
||||
next_version_index = (
|
||||
Document.global_objects.filter(
|
||||
@@ -826,6 +850,7 @@ def split(
|
||||
source_mode: SourceMode = SourceModeChoices.LATEST_VERSION,
|
||||
user: User | None = None,
|
||||
trigger_source: PaperlessTask.TriggerSource = PaperlessTask.TriggerSource.WEB_UI,
|
||||
parent_task_id: str | None = None,
|
||||
) -> Literal["OK"]:
|
||||
logger.info(
|
||||
f"Attempting to split document {doc_ids[0]} into {len(pages)} documents",
|
||||
@@ -869,7 +894,17 @@ def split(
|
||||
).set(headers={"trigger_source": trigger_source}),
|
||||
)
|
||||
|
||||
if delete_originals:
|
||||
if parent_task_id is not None:
|
||||
backup = (
|
||||
release_archive_serial_numbers([doc.id]) if delete_originals else {}
|
||||
)
|
||||
queue_processing_tasks(
|
||||
parent_task_id,
|
||||
consume_tasks,
|
||||
delete_ids=[doc.id] if delete_originals else None,
|
||||
asn_backup=backup,
|
||||
)
|
||||
elif delete_originals:
|
||||
backup = release_archive_serial_numbers([doc.id])
|
||||
logger.info(
|
||||
"Queueing removal of original document after consumption of the split documents",
|
||||
@@ -901,6 +936,7 @@ def delete_pages(
|
||||
source_mode: SourceMode = SourceModeChoices.LATEST_VERSION,
|
||||
user: User | None = None,
|
||||
trigger_source: PaperlessTask.TriggerSource = PaperlessTask.TriggerSource.WEB_UI,
|
||||
parent_task_id: str | None = None,
|
||||
) -> Literal["OK"]:
|
||||
logger.info(
|
||||
f"Attempting to delete pages {pages} from {len(doc_ids)} documents",
|
||||
@@ -919,16 +955,15 @@ def delete_pages(
|
||||
overrides = DocumentMetadataOverrides().from_document(pair.root_doc)
|
||||
if user is not None:
|
||||
overrides.actor_id = user.id
|
||||
consume_file.apply_async(
|
||||
kwargs={
|
||||
"input_doc": ConsumableDocument(
|
||||
source=DocumentSource.ConsumeFolder,
|
||||
original_file=filepath,
|
||||
root_document_id=pair.root_doc.id,
|
||||
),
|
||||
"overrides": overrides,
|
||||
},
|
||||
headers={"trigger_source": trigger_source},
|
||||
queue_consumption(
|
||||
input_doc=ConsumableDocument(
|
||||
source=DocumentSource.ConsumeFolder,
|
||||
original_file=filepath,
|
||||
root_document_id=pair.root_doc.id,
|
||||
),
|
||||
overrides=overrides,
|
||||
trigger_source=trigger_source,
|
||||
parent_task_id=parent_task_id,
|
||||
)
|
||||
logger.info(
|
||||
f"Queued new version for document {pair.root_doc.id} after deleting pages {pages}",
|
||||
@@ -950,6 +985,7 @@ def edit_pdf(
|
||||
source_mode: SourceMode = SourceModeChoices.LATEST_VERSION,
|
||||
user: User | None = None,
|
||||
trigger_source: PaperlessTask.TriggerSource = PaperlessTask.TriggerSource.WEB_UI,
|
||||
parent_task_id: str | None = None,
|
||||
) -> Literal["OK"]:
|
||||
"""
|
||||
Operations is a list of dictionaries describing the final PDF pages.
|
||||
@@ -994,16 +1030,15 @@ def edit_pdf(
|
||||
if user is not None:
|
||||
overrides.owner_id = user.id
|
||||
overrides.actor_id = user.id
|
||||
consume_file.apply_async(
|
||||
kwargs={
|
||||
"input_doc": ConsumableDocument(
|
||||
source=DocumentSource.ConsumeFolder,
|
||||
original_file=filepath,
|
||||
root_document_id=pair.root_doc.id,
|
||||
),
|
||||
"overrides": overrides,
|
||||
},
|
||||
headers={"trigger_source": trigger_source},
|
||||
queue_consumption(
|
||||
input_doc=ConsumableDocument(
|
||||
source=DocumentSource.ConsumeFolder,
|
||||
original_file=filepath,
|
||||
root_document_id=pair.root_doc.id,
|
||||
),
|
||||
overrides=overrides,
|
||||
trigger_source=trigger_source,
|
||||
parent_task_id=parent_task_id,
|
||||
)
|
||||
else:
|
||||
version_filepaths = pdf_ops.build_pdfs(
|
||||
@@ -1043,7 +1078,17 @@ def edit_pdf(
|
||||
).set(headers={"trigger_source": trigger_source}),
|
||||
)
|
||||
|
||||
if delete_original:
|
||||
if parent_task_id is not None:
|
||||
backup = (
|
||||
release_archive_serial_numbers([doc.id]) if delete_original else {}
|
||||
)
|
||||
queue_processing_tasks(
|
||||
parent_task_id,
|
||||
consume_tasks,
|
||||
delete_ids=[doc.id] if delete_original else None,
|
||||
asn_backup=backup,
|
||||
)
|
||||
elif delete_original:
|
||||
backup = release_archive_serial_numbers([doc.id])
|
||||
try:
|
||||
chord(
|
||||
@@ -1077,6 +1122,7 @@ def remove_password(
|
||||
source_mode: SourceMode = SourceModeChoices.LATEST_VERSION,
|
||||
user: User | None = None,
|
||||
trigger_source: PaperlessTask.TriggerSource = PaperlessTask.TriggerSource.WEB_UI,
|
||||
parent_task_id: str | None = None,
|
||||
source_paths_by_id: Mapping[int, Path] | None = None,
|
||||
) -> Literal["OK"]:
|
||||
"""
|
||||
@@ -1119,16 +1165,15 @@ def remove_password(
|
||||
if user is not None:
|
||||
overrides.owner_id = user.id
|
||||
overrides.actor_id = user.id
|
||||
consume_file.apply_async(
|
||||
kwargs={
|
||||
"input_doc": ConsumableDocument(
|
||||
source=DocumentSource.ConsumeFolder,
|
||||
original_file=filepath,
|
||||
root_document_id=pair.root_doc.id,
|
||||
),
|
||||
"overrides": overrides,
|
||||
},
|
||||
headers={"trigger_source": trigger_source},
|
||||
queue_consumption(
|
||||
input_doc=ConsumableDocument(
|
||||
source=DocumentSource.ConsumeFolder,
|
||||
original_file=filepath,
|
||||
root_document_id=pair.root_doc.id,
|
||||
),
|
||||
overrides=overrides,
|
||||
trigger_source=trigger_source,
|
||||
parent_task_id=parent_task_id,
|
||||
)
|
||||
else:
|
||||
consume_tasks = []
|
||||
@@ -1151,7 +1196,19 @@ def remove_password(
|
||||
).set(headers={"trigger_source": trigger_source}),
|
||||
)
|
||||
|
||||
if delete_original:
|
||||
if parent_task_id is not None:
|
||||
backup = (
|
||||
release_archive_serial_numbers([doc.id])
|
||||
if delete_original
|
||||
else {}
|
||||
)
|
||||
queue_processing_tasks(
|
||||
parent_task_id,
|
||||
consume_tasks,
|
||||
delete_ids=[doc.id] if delete_original else None,
|
||||
asn_backup=backup,
|
||||
)
|
||||
elif delete_original:
|
||||
chord(
|
||||
header=consume_tasks,
|
||||
body=delete.si([doc.id]),
|
||||
|
||||
@@ -1,6 +1,5 @@
|
||||
"""Queued document bulk operations."""
|
||||
|
||||
from contextlib import nullcontext
|
||||
from typing import Any
|
||||
|
||||
from auditlog.context import set_actor
|
||||
@@ -90,8 +89,9 @@ def snapshot_field(doc_ids: list[int], field: str) -> dict[int, Any]:
|
||||
return values
|
||||
|
||||
|
||||
@shared_task
|
||||
@shared_task(bind=True)
|
||||
def execute_bulk_operation(
|
||||
self,
|
||||
operation: str,
|
||||
document_ids: list[int],
|
||||
parameters: dict,
|
||||
@@ -113,22 +113,25 @@ def execute_bulk_operation(
|
||||
if operation == "run_workflow" and not user.has_perm("documents.view_workflow"):
|
||||
raise PermissionDenied("Insufficient permissions to run this workflow")
|
||||
|
||||
actor_context = set_actor(user) if settings.AUDIT_LOG_ENABLED else nullcontext()
|
||||
|
||||
if operation not in MODIFIED_FIELDS:
|
||||
execution_parameters = parameters.copy()
|
||||
if (
|
||||
operation in TRIGGER_SOURCE_OPERATIONS | {"reprocess"}
|
||||
and self.request.id is not None
|
||||
):
|
||||
execution_parameters["parent_task_id"] = self.request.id
|
||||
if operation in USER_OPERATIONS:
|
||||
execution_parameters["user"] = user
|
||||
if operation in TRIGGER_SOURCE_OPERATIONS:
|
||||
execution_parameters["trigger_source"] = trigger_source
|
||||
modified_field = LEGACY_MODIFIED_FIELDS.get(operation)
|
||||
with actor_context:
|
||||
with set_actor(user):
|
||||
old_values = (
|
||||
snapshot_field(document_ids, modified_field)
|
||||
if settings.AUDIT_LOG_ENABLED and modified_field
|
||||
else {}
|
||||
)
|
||||
method(document_ids, **execution_parameters)
|
||||
result = method(document_ids, **execution_parameters)
|
||||
if settings.AUDIT_LOG_ENABLED and modified_field:
|
||||
from auditlog.models import LogEntry
|
||||
|
||||
@@ -143,13 +146,17 @@ def execute_bulk_operation(
|
||||
actor=user,
|
||||
additional_data={"reason": f"Bulk edit: {operation}"},
|
||||
)
|
||||
return {"operation": operation, "document_count": len(document_ids)}
|
||||
return {
|
||||
"operation": operation,
|
||||
"document_count": len(document_ids),
|
||||
**(result if isinstance(result, dict) else {}),
|
||||
}
|
||||
|
||||
affected_ids: list[int] = []
|
||||
modified_field = MODIFIED_FIELDS[operation]
|
||||
# Keep metadata writes and their audit entries together. Follow-up processing
|
||||
# runs after commit, within this same tracked task.
|
||||
with actor_context, transaction.atomic():
|
||||
with set_actor(user), transaction.atomic():
|
||||
old_values = (
|
||||
snapshot_field(document_ids, modified_field)
|
||||
if settings.AUDIT_LOG_ENABLED and modified_field
|
||||
|
||||
@@ -3,6 +3,7 @@ from __future__ import annotations
|
||||
import logging
|
||||
import re
|
||||
from fnmatch import fnmatch
|
||||
from fnmatch import translate as fnmatch_translate
|
||||
from typing import TYPE_CHECKING
|
||||
|
||||
from rest_framework import serializers
|
||||
@@ -547,48 +548,6 @@ def existing_document_matches_workflow(
|
||||
return (True, None)
|
||||
|
||||
|
||||
def _glob_to_portable_regex(pattern: str) -> str:
|
||||
"""
|
||||
Translate an fnmatch-style glob into an anchored regex which is accepted by
|
||||
Python, PostgreSQL (ARE), and MariaDB (PCRE).
|
||||
|
||||
fnmatch.translate() can't be used, as its output (`(?s:...)\\Z`) uses syntax
|
||||
PostgreSQL rejects. Supports `*`, `?`, `[seq]` and `[!seq]`.
|
||||
"""
|
||||
out: list[str] = []
|
||||
i = 0
|
||||
n = len(pattern)
|
||||
while i < n:
|
||||
c = pattern[i]
|
||||
i += 1
|
||||
if c == "*":
|
||||
out.append(".*")
|
||||
elif c == "?":
|
||||
out.append(".")
|
||||
elif c == "[":
|
||||
j = i
|
||||
if j < n and pattern[j] == "!":
|
||||
j += 1
|
||||
if j < n and pattern[j] == "]":
|
||||
j += 1
|
||||
while j < n and pattern[j] != "]":
|
||||
j += 1
|
||||
if j >= n:
|
||||
# No closing bracket, treat as a literal
|
||||
out.append(r"\[")
|
||||
else:
|
||||
body = pattern[i:j]
|
||||
i = j + 1
|
||||
negate = body.startswith("!")
|
||||
if negate:
|
||||
body = body[1:]
|
||||
body = "".join(f"\\{ch}" if ch in "\\[^" else ch for ch in body)
|
||||
out.append(f"[{'^' if negate else ''}{body}]")
|
||||
else:
|
||||
out.append(re.escape(c))
|
||||
return f"^{''.join(out)}$"
|
||||
|
||||
|
||||
def prefilter_documents_by_workflowtrigger(
|
||||
documents: QuerySet[Document],
|
||||
trigger: WorkflowTrigger,
|
||||
@@ -669,9 +628,8 @@ def prefilter_documents_by_workflowtrigger(
|
||||
documents = documents.annotate(**annotations).filter(custom_field_q)
|
||||
|
||||
if trigger.filter_filename:
|
||||
documents = documents.filter(
|
||||
original_filename__iregex=_glob_to_portable_regex(trigger.filter_filename),
|
||||
)
|
||||
regex = fnmatch_translate(trigger.filter_filename).lstrip("^").rstrip("$")
|
||||
documents = documents.filter(original_filename__iregex=regex)
|
||||
|
||||
return documents
|
||||
|
||||
|
||||
@@ -0,0 +1,30 @@
|
||||
# Generated by Django 5.2.18 on 2026-11-14 19:59
|
||||
|
||||
import django.db.models.deletion
|
||||
from django.db import migrations
|
||||
from django.db import models
|
||||
|
||||
|
||||
class Migration(migrations.Migration):
|
||||
dependencies = [
|
||||
("documents", "0030_note_format"),
|
||||
]
|
||||
|
||||
operations = [
|
||||
migrations.AddField(
|
||||
model_name="paperlesstask",
|
||||
name="dispatched",
|
||||
field=models.BooleanField(default=False),
|
||||
),
|
||||
migrations.AddField(
|
||||
model_name="paperlesstask",
|
||||
name="parent",
|
||||
field=models.ForeignKey(
|
||||
blank=True,
|
||||
null=True,
|
||||
on_delete=django.db.models.deletion.SET_NULL,
|
||||
related_name="children",
|
||||
to="documents.paperlesstask",
|
||||
),
|
||||
),
|
||||
]
|
||||
@@ -970,6 +970,16 @@ class PaperlessTask(ModelWithOwner):
|
||||
help_text=_("Structured result data from task execution"),
|
||||
)
|
||||
|
||||
parent = models.ForeignKey(
|
||||
"self",
|
||||
null=True,
|
||||
blank=True,
|
||||
on_delete=models.SET_NULL,
|
||||
related_name="children",
|
||||
)
|
||||
|
||||
dispatched = models.BooleanField(default=False)
|
||||
|
||||
# Acknowledgment
|
||||
acknowledged = models.BooleanField(
|
||||
default=False,
|
||||
|
||||
@@ -1717,17 +1717,8 @@ class MergeDocumentsAsVersionsSerializer(DocumentListSerializer):
|
||||
"root_document_id must be one of the selected documents.",
|
||||
)
|
||||
|
||||
# Rechecked under row locks during execution in case document state changes.
|
||||
selected_documents = list(
|
||||
Document.objects.filter(id__in=documents).only("id", "root_document"),
|
||||
)
|
||||
try:
|
||||
bulk_edit.validate_merge_as_versions_state(
|
||||
selected_documents,
|
||||
attrs["root_document_id"],
|
||||
)
|
||||
except ValueError as error:
|
||||
raise serializers.ValidationError(str(error)) from error
|
||||
# Version state checks (e.g., whether a document is already part of a version chain)
|
||||
# moved to merge_as_versions() under row locks, since it could change between queueing and execution
|
||||
return attrs
|
||||
|
||||
|
||||
|
||||
@@ -56,6 +56,8 @@ from documents.models import WorkflowRun
|
||||
from documents.models import WorkflowTrigger
|
||||
from documents.permissions import restrict_queryset_to_visible
|
||||
from documents.plugins.helpers import DocumentsStatusManager
|
||||
from documents.task_tracking import finish_dispatch
|
||||
from documents.task_tracking import update_parent
|
||||
from documents.templating.utils import convert_format_str_to_template_format
|
||||
from documents.utils import compute_checksum
|
||||
from documents.utils import copy_file_with_basic_stats
|
||||
@@ -1077,6 +1079,7 @@ TRACKED_TASKS: dict[str, PaperlessTask.TaskType] = {
|
||||
"documents.tasks.update_document_content_maybe_archive_file": PaperlessTask.TaskType.REPROCESS_DOCUMENT,
|
||||
"documents.tasks.build_share_link_bundle": PaperlessTask.TaskType.BUILD_SHARE_LINK,
|
||||
"documents.bulk_edit.delete": PaperlessTask.TaskType.BULK_DELETE,
|
||||
"documents.task_tracking.finish_file_operation": PaperlessTask.TaskType.BULK_DELETE,
|
||||
"documents.tasks.apply_ai_suggestions": PaperlessTask.TaskType.APPLY_AI_SUGGESTIONS,
|
||||
BULK_OPERATION_TASK_NAME: PaperlessTask.TaskType.BULK_UPDATE,
|
||||
}
|
||||
@@ -1125,6 +1128,8 @@ def _extract_input_data(
|
||||
data["overrides"] = override_dict
|
||||
return data
|
||||
|
||||
if task_type == PaperlessTask.TaskType.REPROCESS_DOCUMENT:
|
||||
return {"document_id": task_kwargs["document_id"]}
|
||||
if task_type == PaperlessTask.TaskType.MAIL_FETCH:
|
||||
account_ids = task_kwargs.get("account_ids")
|
||||
if account_ids is not None:
|
||||
@@ -1212,6 +1217,19 @@ def before_task_publish_handler(
|
||||
input_data = _extract_input_data(task_type, task_kwargs)
|
||||
owner_id = _extract_owner_id(task_type, task_kwargs)
|
||||
trigger_source = _determine_trigger_source(headers)
|
||||
# Consumption can fan out again (e.g. barcode splitting). Keep those
|
||||
# descendants attached so their caller cannot finish before them.
|
||||
parent = None
|
||||
if task_type == PaperlessTask.TaskType.CONSUME_FILE and headers.get(
|
||||
"parent_id",
|
||||
):
|
||||
parent = PaperlessTask.objects.filter(
|
||||
task_id=headers["parent_id"],
|
||||
parent__isnull=False,
|
||||
).first()
|
||||
if parent is not None:
|
||||
owner_id = parent.owner_id
|
||||
trigger_source = parent.trigger_source
|
||||
|
||||
# A retried task is republished with the same task_id, so this fires
|
||||
# again for it; get_or_create keeps the original PENDING record
|
||||
@@ -1224,6 +1242,7 @@ def before_task_publish_handler(
|
||||
"status": PaperlessTask.Status.PENDING,
|
||||
"input_data": input_data,
|
||||
"owner_id": owner_id,
|
||||
"parent": parent,
|
||||
},
|
||||
)
|
||||
except Exception: # pragma: no cover
|
||||
@@ -1309,6 +1328,10 @@ def task_postrun_handler(
|
||||
except PaperlessTask.DoesNotExist:
|
||||
return
|
||||
|
||||
if task_instance.children.exists():
|
||||
finish_dispatch(task_instance, retval if isinstance(retval, dict) else {})
|
||||
return
|
||||
|
||||
task_instance.status = new_status
|
||||
task_instance.date_done = now
|
||||
changed_fields = ["status", "date_done"]
|
||||
@@ -1331,7 +1354,21 @@ def task_postrun_handler(
|
||||
task_instance.status = PaperlessTask.Status.FAILURE
|
||||
changed_fields.append("status")
|
||||
|
||||
if (
|
||||
task_instance.parent_id is not None
|
||||
and task_instance.task_type == PaperlessTask.TaskType.CONSUME_FILE
|
||||
and (not isinstance(retval, dict) or "document_id" not in retval)
|
||||
and not (isinstance(retval, dict) and "duplicate_of" in retval)
|
||||
):
|
||||
task_instance.status = PaperlessTask.Status.FAILURE
|
||||
task_instance.result_data = {
|
||||
**(retval if isinstance(retval, dict) else {}),
|
||||
"error_message": "Consumption stopped without creating a document.",
|
||||
}
|
||||
changed_fields.extend(["status", "result_data"])
|
||||
|
||||
task_instance.save(update_fields=changed_fields)
|
||||
update_parent(task_instance)
|
||||
if task and task.name == BULK_OPERATION_TASK_NAME:
|
||||
_send_bulk_operation_status(task_instance)
|
||||
except Exception: # pragma: no cover
|
||||
@@ -1388,6 +1425,10 @@ def task_failure_handler(
|
||||
}
|
||||
|
||||
task_qs = PaperlessTask.objects.filter(task_id=task_id)
|
||||
parent = task_qs.first()
|
||||
if parent is not None and parent.children.exists():
|
||||
finish_dispatch(parent, result_data)
|
||||
return
|
||||
task_instance = task_qs.values("date_started", "date_created").first()
|
||||
if task_instance:
|
||||
date_started = task_instance["date_started"]
|
||||
@@ -1399,6 +1440,7 @@ def task_failure_handler(
|
||||
date_started - date_created
|
||||
).total_seconds()
|
||||
task_qs.update(**update_fields)
|
||||
update_parent(task_qs.get())
|
||||
if sender and sender.name == BULK_OPERATION_TASK_NAME:
|
||||
_send_bulk_operation_status(task_qs.first())
|
||||
except Exception: # pragma: no cover
|
||||
@@ -1445,6 +1487,8 @@ def task_revoked_handler(
|
||||
"reason": (reason + " Check worker logs for details."),
|
||||
},
|
||||
)
|
||||
if task_instance := task_qs.first():
|
||||
update_parent(task_instance)
|
||||
if sender and sender.name == BULK_OPERATION_TASK_NAME:
|
||||
_send_bulk_operation_status(task_qs.first())
|
||||
except Exception: # pragma: no cover
|
||||
|
||||
@@ -0,0 +1,242 @@
|
||||
"""Completion tracking for operations which dispatch document processing tasks."""
|
||||
|
||||
from __future__ import annotations
|
||||
|
||||
import uuid
|
||||
from typing import TYPE_CHECKING
|
||||
|
||||
from celery import shared_task
|
||||
from django.db import transaction
|
||||
from django.utils import timezone
|
||||
|
||||
from documents.models import PaperlessTask
|
||||
|
||||
if TYPE_CHECKING:
|
||||
from celery.canvas import Signature
|
||||
|
||||
from documents.data_models import ConsumableDocument
|
||||
from documents.data_models import DocumentMetadataOverrides
|
||||
|
||||
|
||||
def queue_consumption(
|
||||
*,
|
||||
input_doc: ConsumableDocument,
|
||||
overrides: DocumentMetadataOverrides,
|
||||
trigger_source: str,
|
||||
parent_task_id: str | None,
|
||||
) -> None:
|
||||
from documents.tasks import consume_file
|
||||
|
||||
kwargs = {"input_doc": input_doc, "overrides": overrides}
|
||||
if parent_task_id is None:
|
||||
consume_file.apply_async(
|
||||
kwargs=kwargs,
|
||||
headers={"trigger_source": trigger_source},
|
||||
)
|
||||
else:
|
||||
queue_processing_tasks(parent_task_id, [consume_file.s(**kwargs)])
|
||||
|
||||
|
||||
def queue_reprocessing_tasks(
|
||||
parent_task_id: str,
|
||||
document_ids: list[int],
|
||||
*,
|
||||
remote_ocr: bool,
|
||||
) -> None:
|
||||
from documents.tasks import update_document_content_maybe_archive_file
|
||||
|
||||
queue_processing_tasks(
|
||||
parent_task_id,
|
||||
[
|
||||
update_document_content_maybe_archive_file.s(
|
||||
document_id=document_id,
|
||||
remote_ocr=remote_ocr,
|
||||
)
|
||||
for document_id in document_ids
|
||||
],
|
||||
)
|
||||
|
||||
|
||||
def queue_processing_tasks(
|
||||
parent_task_id: str,
|
||||
signatures: list[Signature],
|
||||
*,
|
||||
delete_ids: list[int] | None = None,
|
||||
asn_backup: dict | None = None,
|
||||
) -> None:
|
||||
from documents.signals.handlers import TRACKED_TASKS
|
||||
from documents.signals.handlers import _extract_input_data
|
||||
|
||||
parent = PaperlessTask.objects.get(task_id=parent_task_id)
|
||||
children = [
|
||||
PaperlessTask(
|
||||
task_id=str(uuid.uuid4()),
|
||||
task_type=TRACKED_TASKS[signature.task],
|
||||
trigger_source=parent.trigger_source,
|
||||
owner_id=parent.owner_id,
|
||||
parent=parent,
|
||||
input_data=_extract_input_data(
|
||||
TRACKED_TASKS[signature.task],
|
||||
signature.kwargs,
|
||||
),
|
||||
)
|
||||
for signature in signatures
|
||||
]
|
||||
# Dependencies and their final step must be registered together.
|
||||
with transaction.atomic():
|
||||
PaperlessTask.objects.bulk_create(children)
|
||||
if delete_ids:
|
||||
PaperlessTask.objects.create(
|
||||
task_id=str(uuid.uuid4()),
|
||||
task_type=PaperlessTask.TaskType.BULK_DELETE,
|
||||
owner_id=parent.owner_id,
|
||||
trigger_source=parent.trigger_source,
|
||||
parent=parent,
|
||||
input_data={
|
||||
"dependencies": [child.pk for child in children],
|
||||
"delete_ids": delete_ids,
|
||||
"asn_backup": asn_backup or {},
|
||||
},
|
||||
)
|
||||
for index, (signature, child) in enumerate(zip(signatures, children, strict=True)):
|
||||
try:
|
||||
signature.apply_async(
|
||||
task_id=child.task_id,
|
||||
headers={"trigger_source": parent.trigger_source},
|
||||
)
|
||||
except Exception:
|
||||
PaperlessTask.objects.filter(
|
||||
pk__in=[task.pk for task in children[index:]],
|
||||
).update(
|
||||
status=PaperlessTask.Status.FAILURE,
|
||||
date_done=timezone.now(),
|
||||
result_data={"error_message": "Could not publish processing task."},
|
||||
)
|
||||
raise
|
||||
|
||||
|
||||
@shared_task
|
||||
def finish_file_operation(
|
||||
dependencies: list[int],
|
||||
delete_ids: list[int],
|
||||
asn_backup: dict,
|
||||
) -> dict[str, str]:
|
||||
from documents.bulk_edit import delete
|
||||
from documents.bulk_edit import restore_archive_serial_numbers
|
||||
|
||||
statuses = list(
|
||||
PaperlessTask.objects.filter(pk__in=dependencies).values_list(
|
||||
"status",
|
||||
flat=True,
|
||||
),
|
||||
)
|
||||
if len(statuses) != len(dependencies):
|
||||
raise ValueError(
|
||||
"Processing task records are missing; originals were preserved.",
|
||||
)
|
||||
if any(status != PaperlessTask.Status.SUCCESS for status in statuses):
|
||||
restore_archive_serial_numbers(asn_backup)
|
||||
return {"reason": "Originals preserved; archive serial numbers restored."}
|
||||
delete(delete_ids)
|
||||
return {"reason": "Originals moved to trash."}
|
||||
|
||||
|
||||
def _publish_finalizer(task: PaperlessTask) -> None:
|
||||
try:
|
||||
finish_file_operation.apply_async(
|
||||
task_id=task.task_id,
|
||||
kwargs=task.input_data,
|
||||
headers={"trigger_source": task.trigger_source},
|
||||
)
|
||||
except Exception as error:
|
||||
from documents.signals.handlers import task_failure_handler
|
||||
|
||||
task_failure_handler(
|
||||
sender=finish_file_operation,
|
||||
task_id=task.task_id,
|
||||
exception=error,
|
||||
)
|
||||
|
||||
|
||||
def _schedule_finalizers(parent: PaperlessTask) -> None:
|
||||
for task in parent.children.filter(
|
||||
task_type=PaperlessTask.TaskType.BULK_DELETE,
|
||||
status=PaperlessTask.Status.PENDING,
|
||||
):
|
||||
dependencies = task.input_data.get("dependencies")
|
||||
if dependencies is None or task.result_data:
|
||||
continue
|
||||
if (
|
||||
PaperlessTask.objects.filter(pk__in=dependencies)
|
||||
.exclude(
|
||||
status__in=PaperlessTask.COMPLETE_STATUSES,
|
||||
)
|
||||
.exists()
|
||||
):
|
||||
continue
|
||||
task.result_data = {"scheduled": True}
|
||||
task.save(update_fields=["result_data"])
|
||||
transaction.on_commit(lambda task=task: _publish_finalizer(task))
|
||||
|
||||
|
||||
def finish_dispatch(task: PaperlessTask, result_data: dict) -> None:
|
||||
"""Record the scheduling outcome without completing an active parent."""
|
||||
with transaction.atomic():
|
||||
parent = PaperlessTask.objects.select_for_update().get(pk=task.pk)
|
||||
if parent.is_complete:
|
||||
return
|
||||
parent.dispatched = True
|
||||
parent.result_data = result_data
|
||||
parent.save(update_fields=["dispatched", "result_data"])
|
||||
_finalize_parent(parent)
|
||||
|
||||
|
||||
def update_parent(task: PaperlessTask) -> None:
|
||||
if task.parent_id is None:
|
||||
return
|
||||
with transaction.atomic():
|
||||
parent = PaperlessTask.objects.select_for_update().get(pk=task.parent_id)
|
||||
_finalize_parent(parent)
|
||||
|
||||
|
||||
def _finalize_parent(parent: PaperlessTask) -> None:
|
||||
if parent.is_complete or not parent.dispatched:
|
||||
return
|
||||
_schedule_finalizers(parent)
|
||||
statuses = list(parent.children.values_list("status", flat=True))
|
||||
if any(status not in PaperlessTask.COMPLETE_STATUSES for status in statuses):
|
||||
return
|
||||
|
||||
succeeded = statuses.count(PaperlessTask.Status.SUCCESS)
|
||||
failed = len(statuses) - succeeded
|
||||
result_data = parent.result_data or {}
|
||||
if failed and not result_data.get("error_message"):
|
||||
result_data["error_message"] = (
|
||||
f"{succeeded} child task(s) succeeded; {failed} failed or were revoked. "
|
||||
"See child tasks for details."
|
||||
)
|
||||
parent.status = (
|
||||
PaperlessTask.Status.FAILURE
|
||||
if result_data.get("error_message")
|
||||
else PaperlessTask.Status.SUCCESS
|
||||
)
|
||||
parent.result_data = {
|
||||
**parent.input_data,
|
||||
**result_data,
|
||||
"succeeded_count": succeeded,
|
||||
"failed_count": failed,
|
||||
}
|
||||
parent.date_done = timezone.now()
|
||||
if parent.date_started:
|
||||
parent.duration_seconds = (
|
||||
parent.date_done - parent.date_started
|
||||
).total_seconds()
|
||||
parent.wait_time_seconds = (
|
||||
parent.date_started - parent.date_created
|
||||
).total_seconds()
|
||||
parent.save()
|
||||
from documents.signals.handlers import _send_bulk_operation_status
|
||||
|
||||
if "operation" in parent.input_data:
|
||||
transaction.on_commit(lambda: _send_bulk_operation_status(parent))
|
||||
update_parent(parent)
|
||||
@@ -394,11 +394,10 @@ def update_document_content_maybe_archive_file(
|
||||
)
|
||||
|
||||
if not parser_class:
|
||||
logger.error(
|
||||
raise ValueError(
|
||||
f"No parser found for mime type {mime_type}, cannot "
|
||||
f"archive document {document} (ID: {document_id})",
|
||||
f"reprocess document {document} (ID: {document_id})",
|
||||
)
|
||||
return
|
||||
|
||||
with parser_class() as parser:
|
||||
parser.configure(ParserContext())
|
||||
@@ -512,6 +511,7 @@ def update_document_content_maybe_archive_file(
|
||||
logger.exception(
|
||||
f"Error while parsing document {document} (ID: {document_id})",
|
||||
)
|
||||
raise
|
||||
|
||||
|
||||
@shared_task
|
||||
|
||||
@@ -4,18 +4,14 @@ from typing import TYPE_CHECKING
|
||||
from unittest import mock
|
||||
|
||||
import pytest
|
||||
from asgiref.sync import async_to_sync
|
||||
from rest_framework import status
|
||||
from rest_framework.test import APIClient
|
||||
from rest_framework.test import APITestCase
|
||||
|
||||
from documents.views import _iterate_in_sync_thread
|
||||
from paperless_testing.factories import UserFactory
|
||||
from paperless_testing.permissions import grant_global
|
||||
|
||||
if TYPE_CHECKING:
|
||||
from collections.abc import Generator
|
||||
|
||||
from django.contrib.auth.models import User
|
||||
from pytest_mock import MockerFixture
|
||||
|
||||
@@ -66,7 +62,7 @@ class TestChatStreamingViewInputValidation(APITestCase):
|
||||
),
|
||||
mock.patch(
|
||||
"paperless_ai.chat.stream_chat_with_documents",
|
||||
return_value=(c for c in chunks),
|
||||
return_value=iter(chunks),
|
||||
),
|
||||
):
|
||||
resp = self.client.post(
|
||||
@@ -78,11 +74,7 @@ class TestChatStreamingViewInputValidation(APITestCase):
|
||||
|
||||
assert resp.status_code == status.HTTP_200_OK
|
||||
assert not resp.has_header("Content-Encoding")
|
||||
|
||||
async def read_parts() -> list[bytes]:
|
||||
return [part async for part in resp.streaming_content]
|
||||
|
||||
assert async_to_sync(read_parts)() == [c.encode() for c in chunks]
|
||||
assert list(resp.streaming_content) == [c.encode() for c in chunks]
|
||||
|
||||
def test_missing_question_is_rejected(self) -> None:
|
||||
with mock.patch(
|
||||
@@ -163,74 +155,3 @@ class TestChatStreamingViewUnrestrictedFlag:
|
||||
assert (
|
||||
mocked_stream_chat.call_args.kwargs["unrestricted"] is expected_unrestricted
|
||||
)
|
||||
|
||||
|
||||
class TestIterateInSyncThread:
|
||||
@pytest.fixture
|
||||
def events(self) -> list[str]:
|
||||
return []
|
||||
|
||||
@pytest.fixture
|
||||
def chunks(self, events: list[str]) -> Generator[str, None, None]:
|
||||
def generate() -> Generator[str, None, None]:
|
||||
try:
|
||||
for chunk in ("first", "second"):
|
||||
events.append(f"produced {chunk}")
|
||||
yield chunk
|
||||
finally:
|
||||
events.append("closed")
|
||||
|
||||
return generate()
|
||||
|
||||
def test_yields_each_chunk_before_pulling_the_next(
|
||||
self,
|
||||
chunks: Generator[str, None, None],
|
||||
events: list[str],
|
||||
) -> None:
|
||||
"""
|
||||
GIVEN:
|
||||
- A sync generator wrapped for async iteration
|
||||
WHEN:
|
||||
- The wrapper is consumed
|
||||
THEN:
|
||||
- Each chunk is delivered before the next one is generated,
|
||||
rather than the whole generator being drained up front
|
||||
"""
|
||||
|
||||
async def consume() -> None:
|
||||
async for chunk in _iterate_in_sync_thread(chunks):
|
||||
events.append(f"received {chunk}")
|
||||
|
||||
async_to_sync(consume)()
|
||||
|
||||
assert events == [
|
||||
"produced first",
|
||||
"received first",
|
||||
"produced second",
|
||||
"received second",
|
||||
"closed",
|
||||
]
|
||||
|
||||
def test_closing_early_closes_the_source_generator(
|
||||
self,
|
||||
chunks: Generator[str, None, None],
|
||||
events: list[str],
|
||||
) -> None:
|
||||
"""
|
||||
GIVEN:
|
||||
- A sync generator wrapped for async iteration
|
||||
WHEN:
|
||||
- The consumer closes the wrapper after the first chunk, as
|
||||
happens when the client disconnects
|
||||
THEN:
|
||||
- The source generator is closed, so its cleanup runs
|
||||
"""
|
||||
|
||||
async def consume_one() -> None:
|
||||
wrapper = _iterate_in_sync_thread(chunks)
|
||||
await anext(wrapper)
|
||||
await wrapper.aclose()
|
||||
|
||||
async_to_sync(consume_one)()
|
||||
|
||||
assert events == ["produced first", "closed"]
|
||||
Loaded 100 of 164 files, more files were not shown because too many files have changed in this diff.
Show more
Reference in new issue
Block a user