Compare commits

..
Author SHA1 Message Date
Trenton H 56ba15d2e0 Chore: remove unused test helpers
Drops the shared_samples_dir fixture, MailAttachmentMock, _SHORT_TEXT and _make_extract_text, none of which any test used.
2026-11-16 09:15:38 -08:00
Trenton H 919e6838c4 Chore: remove unused GNUPG_HOME setting
Nothing read it. The mail GPG code uses EMAIL_GNUPG_HOME.
2026-11-16 08:01:53 -08:00
Trenton H 1c6a7fe970 Chore: remove unused TASK_WORKERS setting
An alias of CELERY_WORKER_CONCURRENCY that nothing read. PAPERLESS_TASK_WORKERS is unchanged.
2026-11-16 08:01:28 -08:00
Trenton H facc8b07ff Chore: remove unused RtlLanguageException
A leftover from the old tesseract parser, nothing referenced it.
2026-11-16 07:50:51 -08:00
Trenton H 28365eb610 Chore: remove unused ColorField and DeprecatedColors
No serializer used ColorField, and DeprecatedColors was only referenced by it.
2026-11-16 07:38:37 -08:00
Trenton H 02d2da6731 Chore: remove legacy DocumentParser base class
Parsers are resolved through the parser registry and ParserProtocol now, and nothing referenced this class. Its now unused imports are removed too.
2026-11-16 07:37:07 -08:00
Trenton H 56a1ef64c3 Chore: remove unused DocumentViewSet._has_document_permissions
It duplicated has_document_operation_permissions and had no callers.
2026-11-16 07:34:36 -08:00
Trenton H 03e53fd724 Chore: remove unused SanityCheckMessages summary properties
has_global_issues was never referenced and total_issue_count was only asserted in tests. The two private issue counters that fed it are removed as well, and the tests assert on has_error and has_warning instead.
2026-11-16 07:22:22 -08:00
Trenton H e1bd11d0e4 Chore: remove unused Tag.parent_pk
It was an alias of tn_parent_id used only by two test asserts, which now use tn_parent_id directly.
2026-11-16 07:19:12 -08:00
Trenton H 1326378a90 Chore: remove unused is_file_ext_supported
Nothing in production called it, only its own test.
2026-11-16 07:18:53 -08:00
166 changed files with 13750 additions and 29962 deletions

No files matched your search

-6
View File
@@ -285,12 +285,6 @@ The API supports various bulk-editing operations which are executed asynchronous
### Documents
Document bulk operations, including the dedicated document action endpoints, return
`{"result":"OK","task_id":"…"}` when queued. Use `/api/tasks/?task_id={uuid}`
to check the task outcome. Actions that create or reprocess files may start further
consumption or reprocessing tasks and completion of the bulk task means those tasks
were scheduled, not that their file processing has finished.
For bulk operations on documents, use the endpoint `/api/documents/bulk_edit/` which accepts
a json payload of the format:
-92
View File
@@ -1,97 +1,5 @@
# Changelog
## paperless-ngx 3.5.0
### Features / Enhancements
- Chorehancement: move all bulk operations to async hand-off [@shamoon](https://github.com/shamoon) ([#14773](https://github.com/paperless-ngx/paperless-ngx/pull/14773))
- Enhancement: render document notes as markdown [@zwif](https://github.com/zwif) ([#14626](https://github.com/paperless-ngx/paperless-ngx/pull/14626))
- Feature: manual workflow trigger [@shamoon](https://github.com/shamoon) ([#14704](https://github.com/paperless-ngx/paperless-ngx/pull/14704))
- Enhancement: make bulk metadata operations run async [@shamoon](https://github.com/shamoon) ([#14711](https://github.com/paperless-ngx/paperless-ngx/pull/14711))
- Fixhancement: add automatic clearing of sessions [@shamoon](https://github.com/shamoon) ([#14722](https://github.com/paperless-ngx/paperless-ngx/pull/14722))
- Tweak: chat reference tweaks [@shamoon](https://github.com/shamoon) ([#14723](https://github.com/paperless-ngx/paperless-ngx/pull/14723))
- Fixhancement: rework saved view widget queue processing to expected [@shamoon](https://github.com/shamoon) ([#14699](https://github.com/paperless-ngx/paperless-ngx/pull/14699))
### Bug Fixes
- Fix: use a constant password mask and consolidate the no-change check [@stumpylog](https://github.com/stumpylog) ([#14784](https://github.com/paperless-ngx/paperless-ngx/pull/14784))
- Fix: use a PostgreSQL-compatible regex for scheduled workflow filename filters [@stumpylog](https://github.com/stumpylog) ([#14783](https://github.com/paperless-ngx/paperless-ngx/pull/14783))
- Fix: get users for saved view create dialog [@shamoon](https://github.com/shamoon) ([#14769](https://github.com/paperless-ngx/paperless-ngx/pull/14769))
- Fix: Allow AI chat filters over more than 32700 documents [@stumpylog](https://github.com/stumpylog) ([#14754](https://github.com/paperless-ngx/paperless-ngx/pull/14754))
- Fix: Enforce CRLF when writing an email message to the file [@stumpylog](https://github.com/stumpylog) ([#14745](https://github.com/paperless-ngx/paperless-ngx/pull/14745))
- Fix: restore image without archive version thumbnail generation [@shamoon](https://github.com/shamoon) ([#14741](https://github.com/paperless-ngx/paperless-ngx/pull/14741))
- Fix: Blacklist certain OCR user arguments that paperless itself uses [@stumpylog](https://github.com/stumpylog) ([#14734](https://github.com/paperless-ngx/paperless-ngx/pull/14734))
- Fix: remove redundant ui task text filters [@shamoon](https://github.com/shamoon) ([#14736](https://github.com/paperless-ngx/paperless-ngx/pull/14736))
- Fix: fix allowed\_hosts default [@shamoon](https://github.com/shamoon) ([#14735](https://github.com/paperless-ngx/paperless-ngx/pull/14735))
- Fixhancement: add automatic clearing of sessions [@shamoon](https://github.com/shamoon) ([#14722](https://github.com/paperless-ngx/paperless-ngx/pull/14722))
- Fix: two dropdown-related UI fixes [@shamoon](https://github.com/shamoon) ([#14717](https://github.com/paperless-ngx/paperless-ngx/pull/14717))
- Fix: use filtered documents for statistics ASN max [@shamoon](https://github.com/shamoon) ([#14710](https://github.com/paperless-ngx/paperless-ngx/pull/14710))
- Fix: make config view permissions consistent [@shamoon](https://github.com/shamoon) ([#14709](https://github.com/paperless-ngx/paperless-ngx/pull/14709))
- Fix: release the LLM index read lock before streaming the chat answer [@stumpylog](https://github.com/stumpylog) ([#14674](https://github.com/paperless-ngx/paperless-ngx/pull/14674))
- Fix: retrieve once per document chat question [@stumpylog](https://github.com/stumpylog) ([#14673](https://github.com/paperless-ngx/paperless-ngx/pull/14673))
- Fixhancement: rework saved view widget queue processing to expected [@shamoon](https://github.com/shamoon) ([#14699](https://github.com/paperless-ngx/paperless-ngx/pull/14699))
- Fix: apply PDF/A conversion once for mail archives [@shamoon](https://github.com/shamoon) ([#14698](https://github.com/paperless-ngx/paperless-ngx/pull/14698))
- Fix default MFA trusted AMR setting [@shamoon](https://github.com/shamoon) ([#14693](https://github.com/paperless-ngx/paperless-ngx/pull/14693))
- Fix: check descendant tag permissions on delete [@shamoon](https://github.com/shamoon) ([#14687](https://github.com/paperless-ngx/paperless-ngx/pull/14687))
- Fix: lock rows for first-user signup [@shamoon](https://github.com/shamoon) ([#14688](https://github.com/paperless-ngx/paperless-ngx/pull/14688))
- Fix: prevent ID badge wrapping under title on mobile [@shamoon](https://github.com/shamoon) ([#14682](https://github.com/paperless-ngx/paperless-ngx/pull/14682))
### Maintenance
- Chore(deps): Bump the actions group across 1 directory with 17 updates @[dependabot[bot]](https://github.com/apps/dependabot) ([#14691](https://github.com/paperless-ngx/paperless-ngx/pull/14691))
### Dependencies
<details>
<summary>4 changes</summary>
- Chore(deps): Bump the pre-commit-dependencies group with 4 updates @[dependabot[bot]](https://github.com/apps/dependabot) ([#14636](https://github.com/paperless-ngx/paperless-ngx/pull/14636))
- Chore(deps): Bump the actions group across 1 directory with 17 updates @[dependabot[bot]](https://github.com/apps/dependabot) ([#14691](https://github.com/paperless-ngx/paperless-ngx/pull/14691))
- docker(deps): bump astral-sh/uv from 0.13.5-python3.14-trixie-slim to 0.13.7-python3.14-trixie-slim @[dependabot[bot]](https://github.com/apps/dependabot) ([#14635](https://github.com/paperless-ngx/paperless-ngx/pull/14635))
- chore(deps): bump django-allauth to 65.21.1 [@stumpylog](https://github.com/stumpylog) ([#14672](https://github.com/paperless-ngx/paperless-ngx/pull/14672))
</details>
### All App Changes
<details>
<summary>32 changes</summary>
- Chorehancement: move all bulk operations to async hand-off [@shamoon](https://github.com/shamoon) ([#14773](https://github.com/paperless-ngx/paperless-ngx/pull/14773))
- Fix: use a constant password mask and consolidate the no-change check [@stumpylog](https://github.com/stumpylog) ([#14784](https://github.com/paperless-ngx/paperless-ngx/pull/14784))
- Fix: use a PostgreSQL-compatible regex for scheduled workflow filename filters [@stumpylog](https://github.com/stumpylog) ([#14783](https://github.com/paperless-ngx/paperless-ngx/pull/14783))
- Fix: get users for saved view create dialog [@shamoon](https://github.com/shamoon) ([#14769](https://github.com/paperless-ngx/paperless-ngx/pull/14769))
- Enhancement: render document notes as markdown [@zwif](https://github.com/zwif) ([#14626](https://github.com/paperless-ngx/paperless-ngx/pull/14626))
- Feature: manual workflow trigger [@shamoon](https://github.com/shamoon) ([#14704](https://github.com/paperless-ngx/paperless-ngx/pull/14704))
- Fix: Allow AI chat filters over more than 32700 documents [@stumpylog](https://github.com/stumpylog) ([#14754](https://github.com/paperless-ngx/paperless-ngx/pull/14754))
- Fix: Enforce CRLF when writing an email message to the file [@stumpylog](https://github.com/stumpylog) ([#14745](https://github.com/paperless-ngx/paperless-ngx/pull/14745))
- Enhancement: make bulk metadata operations run async [@shamoon](https://github.com/shamoon) ([#14711](https://github.com/paperless-ngx/paperless-ngx/pull/14711))
- Performance: load a cached Hugging Face embedding model without contacting the Hub [@stumpylog](https://github.com/stumpylog) ([#14728](https://github.com/paperless-ngx/paperless-ngx/pull/14728))
- Fix: restore image without archive version thumbnail generation [@shamoon](https://github.com/shamoon) ([#14741](https://github.com/paperless-ngx/paperless-ngx/pull/14741))
- Fix: Blacklist certain OCR user arguments that paperless itself uses [@stumpylog](https://github.com/stumpylog) ([#14734](https://github.com/paperless-ngx/paperless-ngx/pull/14734))
- Fix: remove redundant ui task text filters [@shamoon](https://github.com/shamoon) ([#14736](https://github.com/paperless-ngx/paperless-ngx/pull/14736))
- Fix: fix allowed\_hosts default [@shamoon](https://github.com/shamoon) ([#14735](https://github.com/paperless-ngx/paperless-ngx/pull/14735))
- Fixhancement: add automatic clearing of sessions [@shamoon](https://github.com/shamoon) ([#14722](https://github.com/paperless-ngx/paperless-ngx/pull/14722))
- Tweak: chat reference tweaks [@shamoon](https://github.com/shamoon) ([#14723](https://github.com/paperless-ngx/paperless-ngx/pull/14723))
- Fix: two dropdown-related UI fixes [@shamoon](https://github.com/shamoon) ([#14717](https://github.com/paperless-ngx/paperless-ngx/pull/14717))
- Fix: use filtered documents for statistics ASN max [@shamoon](https://github.com/shamoon) ([#14710](https://github.com/paperless-ngx/paperless-ngx/pull/14710))
- Fix: make config view permissions consistent [@shamoon](https://github.com/shamoon) ([#14709](https://github.com/paperless-ngx/paperless-ngx/pull/14709))
- Performance: stop embedding every document ID in selection\_data queries [@stumpylog](https://github.com/stumpylog) ([#14597](https://github.com/paperless-ngx/paperless-ngx/pull/14597))
- Performance: replace django-treenode with adjacency-list tags [@shamoon](https://github.com/shamoon) ([#14612](https://github.com/paperless-ngx/paperless-ngx/pull/14612))
- Performance: use small vec0 chunks and larger pages for the LLM index [@stumpylog](https://github.com/stumpylog) ([#14678](https://github.com/paperless-ngx/paperless-ngx/pull/14678))
- Fix: release the LLM index read lock before streaming the chat answer [@stumpylog](https://github.com/stumpylog) ([#14674](https://github.com/paperless-ngx/paperless-ngx/pull/14674))
- Fix: retrieve once per document chat question [@stumpylog](https://github.com/stumpylog) ([#14673](https://github.com/paperless-ngx/paperless-ngx/pull/14673))
- Fixhancement: rework saved view widget queue processing to expected [@shamoon](https://github.com/shamoon) ([#14699](https://github.com/paperless-ngx/paperless-ngx/pull/14699))
- Fix: apply PDF/A conversion once for mail archives [@shamoon](https://github.com/shamoon) ([#14698](https://github.com/paperless-ngx/paperless-ngx/pull/14698))
- Fix default MFA trusted AMR setting [@shamoon](https://github.com/shamoon) ([#14693](https://github.com/paperless-ngx/paperless-ngx/pull/14693))
- Fix: check descendant tag permissions on delete [@shamoon](https://github.com/shamoon) ([#14687](https://github.com/paperless-ngx/paperless-ngx/pull/14687))
- Fix: lock rows for first-user signup [@shamoon](https://github.com/shamoon) ([#14688](https://github.com/paperless-ngx/paperless-ngx/pull/14688))
- Fix: prevent ID badge wrapping under title on mobile [@shamoon](https://github.com/shamoon) ([#14682](https://github.com/paperless-ngx/paperless-ngx/pull/14682))
- chore(deps): bump django-allauth to 65.21.1 [@stumpylog](https://github.com/stumpylog) ([#14672](https://github.com/paperless-ngx/paperless-ngx/pull/14672))
- chore(deps): bump dev tooling [@stumpylog](https://github.com/stumpylog) ([#14671](https://github.com/paperless-ngx/paperless-ngx/pull/14671))
</details>
## paperless-ngx 3.4.2
### Bug Fixes
+1 -1
View File
@@ -1,6 +1,6 @@
[project]
name = "paperless-ngx"
version = "3.5.0"
version = "3.4.2"
description = """\
A community-supported supercharged document management system: scan, index and archive all your physical documents\
"""
+247 -488
View File
File diff suppressed because it is too large. Load diff
+1 -1
View File
@@ -1,6 +1,6 @@
{
"name": "paperless-ngx-ui",
"version": "3.5.0",
"version": "3.4.2",
"scripts": {
"preinstall": "npx only-allow pnpm",
"ng": "ng",
@@ -175,13 +175,7 @@
</div>
</td>
<td class="overflow-auto name-col">
@let documentCount = taskDocumentCount(task);
<div>
{{ taskDisplayName(task) }}
@if (documentCount !== null && documentCount > 1) {
<small class="text-muted ms-1" i18n>({documentCount, plural, =1 {1 document} other {{{ documentCount }} documents}})</small>
}
</div>
<div>{{ taskDisplayName(task) }}</div>
<div class="small text-muted">
@if (taskShowsSeparateTypeLabel(task)) {
<span class="me-2">{{ task.task_type_display }}</span>
@@ -895,117 +895,6 @@ describe('TasksComponent', () => {
expect(sanityTaskRow.nativeElement.textContent).toContain('System')
})
it.each([
['modify_tags', 12, 'Bulk operation: Update tags', '(12 documents)'],
['run_workflow', 1, 'Run workflow', null],
['delete', 3, 'Bulk operation: Move to trash', '(3 documents)'],
['reprocess', 2, 'Bulk operation: Document reprocessing', '(2 documents)'],
])(
'shows the operation, document count and outcome for %s',
(operation, count, label, summary) => {
const task: PaperlessTask = {
...tasks[4],
task_type:
operation === 'delete'
? PaperlessTaskType.BulkDelete
: PaperlessTaskType.BulkUpdate,
task_type_display:
operation === 'delete' ? 'Bulk Delete' : 'Bulk Update',
trigger_source: PaperlessTaskTriggerSource.WebUI,
trigger_source_display: 'Web UI',
input_data: { operation, document_count: count },
result_data: { operation, document_count: count },
}
component.setSection(TaskSection.Completed)
flushTaskLists(25, 1, [task])
fixture.detectChanges()
const name = fixture.nativeElement.querySelector('tbody .name-col')
expect(name.textContent).toContain(label)
const countLabel = name.querySelector('small.text-muted')
if (summary) {
expect(countLabel.textContent).toContain(summary)
} else {
expect(countLabel).toBeNull()
expect(name.textContent).not.toContain('Bulk operation:')
}
expect(name.textContent).not.toContain('·')
expect(name.textContent).toContain('Web UI')
expect(name.textContent).not.toContain(task.task_type_display)
expect(
name.closest('tr').querySelector('.results-col').textContent
).toContain(
[
'reprocess',
'rotate',
'merge',
'split',
'delete_pages',
'edit_pdf',
'remove_password',
].includes(operation)
? 'Submitted for processing'
: 'Completed'
)
}
)
it('shows operation context while work is pending without reporting completion', () => {
const task = {
...tasks[2],
task_type: PaperlessTaskType.BulkUpdate,
task_type_display: 'Bulk Update',
input_data: { operation: 'reprocess', document_count: 1 },
}
component.setSection(TaskSection.InProgress)
flushTaskLists(25, 1, [task])
fixture.detectChanges()
const name = fixture.nativeElement.querySelector('tbody .name-col')
expect(name.textContent).toContain('Document reprocessing')
expect(name.textContent).not.toContain('Bulk operation:')
expect(name.querySelector('small')).toBeNull()
expect(component.taskResultMessage(task)).toBeNull()
})
it.each([{}, { operation: 'future_operation' }])(
'preserves the fallback for legacy or unknown operation data: %j',
(input_data) => {
const task = {
...tasks[4],
task_type: PaperlessTaskType.BulkUpdate,
task_type_display: 'Bulk Update',
input_data,
result_data: {},
}
expect(component.taskDisplayName(task)).toBe('Bulk Update')
expect(component.taskDocumentCount(task)).toBeNull()
expect(component.taskResultMessage(task)).toBeNull()
}
)
it.each([-1, 1.5, '2', null, undefined])(
'does not display an invalid document count: %s',
(document_count) => {
expect(
component.taskDocumentCount({
...tasks[4],
input_data: { operation: 'modify_tags', document_count },
})
).toBeNull()
}
)
it('retains error details for failed operation tasks', () => {
expect(
component.taskResultMessage({
...tasks[0],
input_data: { operation: 'modify_tags', document_count: 12 },
result_data: { error_message: 'Insufficient permissions' },
})
).toBe('Insufficient permissions')
})
it('should filter tasks by result', () => {
component.setSection(TaskSection.NeedsAttention)
component.filterTargetID = 1
@@ -33,10 +33,6 @@ import { IfPermissionsDirective } from 'src/app/directives/if-permissions.direct
import { CustomDatePipe } from 'src/app/pipes/custom-date.pipe'
import { TasksService } from 'src/app/services/tasks.service'
import { ToastService } from 'src/app/services/toast.service'
import {
documentOperationHasFollowUpTasks,
getDocumentOperationLabel,
} from 'src/app/utils/document-operation'
import { ConfirmDialogComponent } from '../../common/confirm-dialog/confirm-dialog.component'
import { PageHeaderComponent } from '../../common/page-header/page-header.component'
import { LoadingComponentWithPermissions } from '../../loading-component/loading.component'
@@ -418,30 +414,11 @@ export class TasksComponent
}
taskDisplayName(task: PaperlessTask): string {
const filename = task.input_data?.filename?.toString()
if (filename) return filename
const operation = this.taskOperationLabel(task)
const count = this.taskDocumentCount(task)
if (operation && count !== null && count > 1) {
return $localize`Bulk operation: ${operation}`
}
return operation || task.task_type_display
return task.input_data?.filename?.toString() || task.task_type_display
}
taskShowsSeparateTypeLabel(task: PaperlessTask): boolean {
return (
!this.taskOperationLabel(task) &&
this.taskDisplayName(task) !== task.task_type_display
)
}
private taskOperationLabel(task: PaperlessTask): string | null {
return getDocumentOperationLabel(task.input_data?.operation)
}
taskDocumentCount(task: PaperlessTask): number | null {
const count = task.input_data?.document_count
return Number.isInteger(count) && count >= 0 ? count : null
return this.taskDisplayName(task) !== task.task_type_display
}
taskResultMessage(task: PaperlessTask): string | null {
@@ -469,15 +446,6 @@ export class TasksComponent
return errorMessage
}
if (
task.status === PaperlessTaskStatus.Success &&
this.taskOperationLabel(task)
) {
return documentOperationHasFollowUpTasks(task.input_data?.operation)
? $localize`Submitted for processing`
: $localize`Completed`
}
return null
}
@@ -33,7 +33,6 @@ import { RemoteVersionService } from 'src/app/services/rest/remote-version.servi
import { SavedViewService } from 'src/app/services/rest/saved-view.service'
import { SearchService } from 'src/app/services/rest/search.service'
import { SettingsService } from 'src/app/services/settings.service'
import { TaskNotificationService } from 'src/app/services/task-notification.service'
import { TasksService } from 'src/app/services/tasks.service'
import { ToastService } from 'src/app/services/toast.service'
import { UploadDocumentsService } from 'src/app/services/upload-documents.service'
@@ -199,14 +198,14 @@ describe('AppFrameComponent', () => {
fixture.detectChanges()
})
it('refreshes and updates the task notification on completion', () => {
it('refreshes and notifies on completion without client-side task tracking', () => {
const list = TestBed.inject(DocumentListViewService)
const reload = jest.spyOn(list, 'reload').mockImplementation(() => {})
jest.spyOn(list, 'reduceSelectionToFilter').mockImplementation(() => {})
const tasksReload = jest
.spyOn(tasksService, 'reload')
.mockImplementation(() => {})
const success = jest.spyOn(toastService, 'show')
const success = jest.spyOn(toastService, 'showSuccess')
const refresh = jest
.spyOn(openDocumentsService, 'refreshDocument')
.mockImplementation(() => {})
@@ -231,12 +230,7 @@ describe('AppFrameComponent', () => {
expect(refresh).toHaveBeenCalledWith(document.id)
expect(maybeRefreshSpy).toHaveBeenCalledTimes(1)
expect(tasksReload).toHaveBeenCalledTimes(1)
expect(success).toHaveBeenCalledWith(
expect.objectContaining({
id: 'task-bulk-task',
content: "Operation 'Set permissions' completed.",
})
)
expect(success).toHaveBeenCalledWith('Bulk operation completed.')
fixture.destroy()
bulkEvents.next({
task_id: 'bulk-task',
@@ -251,14 +245,14 @@ describe('AppFrameComponent', () => {
const reload = jest.spyOn(list, 'reload').mockImplementation(() => {})
jest.spyOn(list, 'reduceSelectionToFilter').mockImplementation(() => {})
jest.spyOn(tasksService, 'reload').mockImplementation(() => {})
const error = jest.spyOn(toastService, 'show')
const error = jest.spyOn(toastService, 'showError')
const success = jest.spyOn(toastService, 'showSuccess')
for (const status of [
PaperlessTaskStatus.Failure,
PaperlessTaskStatus.Revoked,
]) {
bulkEvents.next({
task_id: `${status}-task`,
task_id: 'bulk-task',
operation: 'set_permissions',
status,
error:
@@ -268,36 +262,11 @@ describe('AppFrameComponent', () => {
})
}
expect(reload).toHaveBeenCalledTimes(2)
expect(error).toHaveBeenCalledWith(expect.any(String), 'Could not update')
expect(error).toHaveBeenCalledWith(
expect.objectContaining({
content:
"Operation 'Set permissions' failed. Some changes may already have been applied.",
error: 'Could not update',
})
'Bulk operation revoked.',
'Task expired before execution.'
)
expect(error).toHaveBeenCalledWith(
expect.objectContaining({
content: "Operation 'Set permissions' revoked.",
error: 'Task expired before execution.',
})
)
expect(success).not.toHaveBeenCalled()
})
it('passes delete completion to the shared task notification handler', () => {
const success = jest.spyOn(toastService, 'showSuccess')
const notifications = TestBed.inject(TaskNotificationService)
const handleStatus = jest.spyOn(notifications, 'complete')
bulkEvents.next({
task_id: 'delete-1',
operation: 'delete',
status: PaperlessTaskStatus.Success,
})
expect(handleStatus).toHaveBeenCalledWith({
task_id: 'delete-1',
operation: 'delete',
status: PaperlessTaskStatus.Success,
})
expect(success).not.toHaveBeenCalled()
})
@@ -57,7 +57,6 @@ import {
} from 'src/app/services/rest/remote-version.service'
import { SavedViewService } from 'src/app/services/rest/saved-view.service'
import { SettingsService } from 'src/app/services/settings.service'
import { TaskNotificationService } from 'src/app/services/task-notification.service'
import { TasksService } from 'src/app/services/tasks.service'
import { ToastService } from 'src/app/services/toast.service'
import { UploadDocumentsService } from 'src/app/services/upload-documents.service'
@@ -116,7 +115,6 @@ export class AppFrameComponent
settingsService = inject(SettingsService)
tasksService = inject(TasksService)
private readonly toastService = inject(ToastService)
private readonly taskNotifications = inject(TaskNotificationService)
private modalService = inject(NgbModal)
permissionsService = inject(PermissionsService)
private djangoMessagesService = inject(DjangoMessagesService)
@@ -184,11 +182,9 @@ export class AppFrameComponent
),
takeUntilDestroyed(this.destroyRef)
)
.subscribe((result) => {
.subscribe(({ status, error }) => {
this.documentList.reload(null, false)
if (!this.documentList.allSelected) {
this.documentList.reduceSelectionToFilter()
}
this.documentList.reduceSelectionToFilter()
// maybe someday we can optimize to only refresh affected documents
this.openDocumentsService.getOpenDocuments().forEach((doc) => {
this.openDocumentsService.refreshDocument(doc.id)
@@ -202,7 +198,16 @@ export class AppFrameComponent
) {
this.tasksService.reload()
}
this.taskNotifications.complete(result)
if (status === PaperlessTaskStatus.Success) {
this.toastService.showSuccess($localize`Bulk operation completed.`)
} else if (status === PaperlessTaskStatus.Revoked) {
this.toastService.showError($localize`Bulk operation revoked.`, error)
} else {
this.toastService.showError(
$localize`Bulk operation failed. Some changes may already have been applied.`,
error
)
}
})
this.lastScrollY = window.scrollY
@@ -7,7 +7,6 @@
background-color: var(--pngx-toast-bg);
color: var(--bs-body-color);
overflow: hidden;
transition: background-color .18s ease, border-color .18s ease;
&.toast-success {
--pngx-toast-accent: var(--pngx-toast-success);
@@ -24,13 +23,6 @@
.toast-icon {
color: var(--pngx-toast-accent);
transition: color .18s ease;
}
@media (prefers-reduced-motion: reduce) {
.toast, .toast-icon {
transition: none;
}
}
.toast-content {
@@ -65,36 +65,6 @@ describe('ToastComponent', () => {
expect(component.delayRemaining()).toEqual(3000)
})
it('updates its color and icon in place and restarts the countdown', () => {
jest.useFakeTimers()
fixture.componentRef.setInput('toast', {
id: 'task-1',
content: 'Queued',
delay: 5000,
})
fixture.detectChanges()
component.onShown(component.toast)
jest.advanceTimersByTime(4500)
const element = fixture.nativeElement.querySelector('ngb-toast')
fixture.componentRef.setInput('toast', {
id: 'task-1',
content: 'Completed',
delay: 5000,
level: ToastLevel.Success,
})
fixture.detectChanges()
expect(fixture.nativeElement.querySelector('ngb-toast')).toBe(element)
expect(element.classList).toContain('toast-success')
expect(element.querySelector('[name="check-circle"]')).not.toBeNull()
expect(component.delayRemaining()).toBe(5000)
const hide = jest.spyOn(component.ngbToast, 'hide')
jest.advanceTimersByTime(1000)
expect(hide).not.toHaveBeenCalled()
jest.advanceTimersByTime(4200)
expect(hide).toHaveBeenCalled()
jest.useRealTimers()
})
it('should hide the toast when the countdown ends', () => {
jest.useFakeTimers()
const toast = { content: 'Info', delay: 3000 }
@@ -5,10 +5,8 @@ import {
DestroyRef,
EventEmitter,
Input,
OnChanges,
OnInit,
Output,
SimpleChanges,
ViewChild,
inject,
signal,
@@ -36,10 +34,10 @@ import { Toast, ToastLevel } from 'src/app/services/toast.service'
styleUrl: './toast.component.scss',
providers: [DatePipe, CustomDatePipe],
})
export class ToastComponent implements OnInit, OnChanges {
private readonly clipboard = inject(Clipboard)
private readonly customDatePipe = inject(CustomDatePipe)
private readonly destroyRef = inject(DestroyRef)
export class ToastComponent implements OnInit {
private clipboard = inject(Clipboard)
private customDatePipe = inject(CustomDatePipe)
private destroyRef = inject(DestroyRef)
readonly copied = signal(false)
readonly ToastLevel = ToastLevel
readonly delayRemaining = signal<number>(null)
@@ -71,12 +69,6 @@ export class ToastComponent implements OnInit, OnChanges {
if (this.autohide) this.delayRemaining.set(this.toast.delay)
}
ngOnChanges(changes: SimpleChanges): void {
if (changes.toast && !changes.toast.firstChange && this.autohide) {
this.onShown(this.toast)
}
}
get relativeTimestamp(): string {
return this.customDatePipe.transform(this.toast.timestamp, 'relative')
}
@@ -68,9 +68,9 @@ describe('ToastsComponent', () => {
toasts.forEach((t) => toastSubject.next(t))
expect(component.toasts().map((t) => t.id)).toEqual(['3', '2', '1'])
// Updating a visible toast preserves its position without duplicating it.
// showing a visible toast again moves it to the top without duplicating
toastSubject.next(toasts[1])
expect(component.toasts().map((t) => t.id)).toEqual(['3', '2', '1'])
expect(component.toasts().map((t) => t.id)).toEqual(['1', '3', '2'])
})
it('should remove a popup when it hides without closing it', () => {
@@ -32,11 +32,10 @@ export class ToastsComponent {
return
}
this.toasts.update((toasts) =>
toasts.some((existing) => existing.id === toast.id)
? toasts.map((existing) =>
existing.id === toast.id ? toast : existing
)
: [toast, ...toasts].slice(0, MAX_VISIBLE_TOASTS)
[toast, ...toasts.filter((t) => t.id !== toast.id)].slice(
0,
MAX_VISIBLE_TOASTS
)
)
})
}
@@ -1013,45 +1013,30 @@ describe('DocumentDetailComponent', () => {
])
})
it('should queue reprocess, show the task link and close the modal', () => {
jest.useFakeTimers()
it('should support reprocess, confirm and close modal after started', () => {
initNormally()
const reprocessSpy = jest.spyOn(documentService, 'reprocessDocuments')
reprocessSpy.mockReturnValue(of({ result: 'OK', task_id: 'reprocess-1' }))
reprocessSpy.mockReturnValue(of(true))
let openModal: NgbModalRef
modalService.activeInstances.subscribe((modal) => (openModal = modal[0]))
const modalSpy = jest.spyOn(modalService, 'open')
const toastSpy = jest.spyOn(toastService, 'show')
const toastSpy = jest.spyOn(toastService, 'showInfo')
component.reprocess()
const modalCloseSpy = jest.spyOn(openModal, 'close')
openModal.componentInstance.confirmClicked.next()
expect(reprocessSpy).toHaveBeenCalledWith({ documents: [doc.id] }, false)
expect(modalSpy).toHaveBeenCalled()
expect(toastSpy).not.toHaveBeenCalled()
expect(toastSpy).toHaveBeenCalled()
expect(modalCloseSpy).toHaveBeenCalled()
jest.advanceTimersByTime(400)
expect(toastSpy).toHaveBeenCalledWith(
expect.objectContaining({
content: 'Document reprocessing queued.',
actionName: 'View task',
})
)
toastSpy.mock.calls.at(-1)[0].action()
expect(router.navigate).toHaveBeenCalledWith(['/tasks'], {
queryParams: { task_id: 'reprocess-1' },
})
expect(modalCloseSpy).toHaveBeenCalled()
jest.useRealTimers()
})
it('should run a workflow on the document, confirm and close modal', () => {
jest.useFakeTimers()
initNormally()
const bulkEditSpy = jest.spyOn(documentService, 'bulkEdit')
bulkEditSpy.mockReturnValue(of({ result: 'OK', task_id: 'workflow-1' }))
bulkEditSpy.mockReturnValue(of(true))
let openModal: NgbModalRef
modalService.activeInstances.subscribe((modal) => (openModal = modal[0]))
const toastSpy = jest.spyOn(toastService, 'show')
const toastSpy = jest.spyOn(toastService, 'showInfo')
component.runWorkflow()
expect(openModal.componentInstance.singleDocument).toBe(true)
expect(openModal.componentInstance.force).toBe(true)
@@ -1063,18 +1048,14 @@ describe('DocumentDetailComponent', () => {
'run_workflow',
{ workflow: 5, filter_documents: false }
)
jest.advanceTimersByTime(400)
expect(toastSpy).toHaveBeenCalledWith(
expect.objectContaining({ content: 'Workflow queued.' })
)
expect(toastSpy).toHaveBeenCalled()
expect(modalCloseSpy).toHaveBeenCalled()
jest.useRealTimers()
})
it('should apply workflow filters when the single-document checkbox is cleared', () => {
initNormally()
const bulkEditSpy = jest.spyOn(documentService, 'bulkEdit')
bulkEditSpy.mockReturnValue(of({ result: 'OK', task_id: 'workflow-2' }))
bulkEditSpy.mockReturnValue(of(true))
let openModal: NgbModalRef
modalService.activeInstances.subscribe((modal) => (openModal = modal[0]))
component.runWorkflow()
@@ -1091,7 +1072,7 @@ describe('DocumentDetailComponent', () => {
it('should pass remote OCR choice when reprocessing', () => {
initNormally()
const reprocessSpy = jest.spyOn(documentService, 'reprocessDocuments')
reprocessSpy.mockReturnValue(of({ result: 'OK', task_id: 'reprocess-2' }))
reprocessSpy.mockReturnValue(of(true))
let openModal: NgbModalRef
modalService.activeInstances.subscribe((modal) => (openModal = modal[0]))
component.reprocess()
@@ -2020,12 +2001,10 @@ describe('DocumentDetailComponent', () => {
})
it('should support pdf editor, handle error', () => {
jest.useFakeTimers()
let modal: NgbModalRef
modalService.activeInstances.subscribe((m) => (modal = m[0]))
const closeSpy = jest.spyOn(openDocumentsService, 'closeDocument')
const errorSpy = jest.spyOn(toastService, 'showError')
const queuedToastSpy = jest.spyOn(toastService, 'show')
initNormally()
component.selectedVersionId.set(10)
component.editPdf()
@@ -2044,7 +2023,6 @@ describe('DocumentDetailComponent', () => {
update_document: false,
include_metadata: true,
source_mode: 'explicit_selection',
from_webui: true,
})
req.error(new ErrorEvent('failed'))
expect(errorSpy).toHaveBeenCalled()
@@ -2057,21 +2035,13 @@ describe('DocumentDetailComponent', () => {
req = httpTestingController.expectOne(
`${environment.apiBaseUrl}documents/edit_pdf/`
)
req.flush({ result: 'OK', task_id: 'edit-pdf-1' })
jest.advanceTimersByTime(400)
expect(closeSpy).not.toHaveBeenCalled()
expect(queuedToastSpy).toHaveBeenCalledWith(
expect.objectContaining({ content: 'PDF edit queued.' })
)
jest.useRealTimers()
req.flush(true)
expect(closeSpy).toHaveBeenCalled()
})
it('should support removing password protection from pdfs', () => {
jest.useFakeTimers()
let modal: NgbModalRef
modalService.activeInstances.subscribe((m) => (modal = m[0]))
const closeSpy = jest.spyOn(openDocumentsService, 'closeDocument')
const queuedToastSpy = jest.spyOn(toastService, 'show')
initNormally()
component.selectedVersionId.set(10)
component.password = 'secret'
@@ -2092,15 +2062,8 @@ describe('DocumentDetailComponent', () => {
include_metadata: false,
delete_original: true,
source_mode: 'explicit_selection',
from_webui: true,
})
req.flush({ result: 'OK', task_id: 'remove-password-1' })
jest.advanceTimersByTime(400)
expect(closeSpy).not.toHaveBeenCalled()
expect(queuedToastSpy).toHaveBeenCalledWith(
expect.objectContaining({ content: 'Password removal queued.' })
)
jest.useRealTimers()
req.flush(true)
})
it('should require the current password before removing it', () => {
@@ -2152,9 +2115,9 @@ describe('DocumentDetailComponent', () => {
const req = httpTestingController.expectOne(
`${environment.apiBaseUrl}documents/remove_password/`
)
req.flush({ result: 'OK', task_id: 'remove-password-2' })
req.flush(true)
expect(refreshSpy).not.toHaveBeenCalled()
expect(refreshSpy).toHaveBeenCalledWith(doc.id)
})
it('should support keyboard shortcuts', () => {
@@ -97,7 +97,6 @@ import { TagService } from 'src/app/services/rest/tag.service'
import { UserService } from 'src/app/services/rest/user.service'
import { WorkflowService } from 'src/app/services/rest/workflow.service'
import { SettingsService } from 'src/app/services/settings.service'
import { TaskNotificationService } from 'src/app/services/task-notification.service'
import { ToastService } from 'src/app/services/toast.service'
import { TrashService } from 'src/app/services/trash.service'
import { WebsocketStatusService } from 'src/app/services/websocket-status.service'
@@ -227,7 +226,6 @@ export class DocumentDetailComponent
private documentListViewService = inject(DocumentListViewService)
private documentTitlePipe = inject(DocumentTitlePipe)
private toastService = inject(ToastService)
private taskNotifications = inject(TaskNotificationService)
private trashService = inject(TrashService)
private settings = inject(SettingsService)
private storagePathService = inject(StoragePathService)
@@ -1560,11 +1558,9 @@ export class DocumentDetailComponent
modal.componentInstance.remoteOcr
)
.subscribe({
next: ({ task_id }) => {
this.taskNotifications.showQueued(
task_id,
$localize`Document reprocessing queued.`,
{ content: $localize`Document submitted for reprocessing.` }
next: () => {
this.toastService.showInfo(
$localize`Reprocess operation for "${this.document().title}" will begin in the background.`
)
if (modal) {
modal.close()
@@ -1601,13 +1597,9 @@ export class DocumentDetailComponent
filter_documents: !modal.componentInstance.force,
})
.subscribe({
next: ({ task_id }) => {
this.taskNotifications.showQueued(
task_id,
$localize`Workflow queued.`,
{
content: $localize`Workflow completed.`,
}
next: () => {
this.toastService.showInfo(
$localize`Workflow will run on "${this.document().title}" in the background.`
)
modal.close()
},
@@ -1997,15 +1989,14 @@ export class DocumentDetailComponent
})
.pipe(first(), takeUntil(this.unsubscribeNotifier))
.subscribe({
next: ({ task_id }) => {
this.taskNotifications.showQueued(
task_id,
$localize`PDF edit queued.`,
{
content: $localize`PDF edit submitted for processing.`,
}
next: () => {
this.toastService.showInfo(
$localize`PDF edit operation for "${this.document().title}" will begin in the background.`
)
modal.close()
if (modal.componentInstance.deleteOriginal) {
this.openDocumentService.closeDocument(this.document())
}
},
error: (error) => {
if (modal) {
@@ -2055,16 +2046,17 @@ export class DocumentDetailComponent
})
.pipe(first(), takeUntil(this.unsubscribeNotifier))
.subscribe({
next: ({ task_id }) => {
this.taskNotifications.showQueued(
task_id,
$localize`Password removal queued.`,
{
content: $localize`Password removal submitted for processing.`,
}
next: () => {
this.toastService.showInfo(
$localize`Password removal operation for "${this.document().title}" will begin in the background.`
)
this.networkActive.set(false)
modal.close()
if (!dialog.updateDocument && dialog.deleteOriginal) {
this.openDocumentService.closeDocument(this.document())
} else if (dialog.updateDocument) {
this.openDocumentService.refreshDocument(this.documentId())
}
},
error: (error) => {
dialog.buttonsEnabled.set(true)
@@ -2,7 +2,6 @@ import { DatePipe } from '@angular/common'
import { SimpleChange, signal } from '@angular/core'
import { ComponentFixture, TestBed } from '@angular/core/testing'
import { By } from '@angular/platform-browser'
import { Router } from '@angular/router'
import { NgbModal } from '@ng-bootstrap/ng-bootstrap'
import { Subject, of, throwError } from 'rxjs'
import { DocumentVersionInfo } from 'src/app/data/document'
@@ -34,7 +33,7 @@ describe('DocumentVersionDropdownComponent', () => {
>
>
let toastService: jest.Mocked<
Pick<ToastService, 'show' | 'showError' | 'showInfo' | 'showSuccess'>
Pick<ToastService, 'showError' | 'showInfo' | 'showSuccess'>
>
let finished$: Subject<{ taskId: string }>
let failed$: Subject<{ taskId: string; message?: string }>
@@ -55,7 +54,6 @@ describe('DocumentVersionDropdownComponent', () => {
}
modalService = { open: jest.fn() }
toastService = {
show: jest.fn(),
showError: jest.fn(),
showInfo: jest.fn(),
showSuccess: jest.fn(),
@@ -90,10 +88,6 @@ describe('DocumentVersionDropdownComponent', () => {
provide: PermissionsService,
useValue: permissionsService,
},
{
provide: Router,
useValue: { navigate: jest.fn() },
},
{
provide: WebsocketStatusService,
useValue: {
@@ -383,8 +377,7 @@ describe('DocumentVersionDropdownComponent', () => {
expect(component.versionLabelDraft).toEqual('')
})
it('addExistingDocumentAsVersion should queue with a label and wait to refresh versions', () => {
jest.useFakeTimers()
it('addExistingDocumentAsVersion should merge with a label and refresh versions', () => {
const confirmClicked = new Subject<number>()
const modal = {
componentInstance: {
@@ -395,9 +388,14 @@ describe('DocumentVersionDropdownComponent', () => {
close: jest.fn(),
}
modalService.open.mockReturnValue(modal as any)
documentService.mergeDocumentsAsVersions.mockReturnValue(
of({ result: 'OK', task_id: 'versions-1' })
)
documentService.mergeDocumentsAsVersions.mockReturnValue(of({} as any))
// Newest first, as the API returns them. The merged document has a lower id
// than the root, which is the whole point of merging an existing document.
const versions: DocumentVersionInfo[] = [
{ id: 2, is_root: false, checksum: 'cccc' },
{ id: 3, is_root: true, checksum: 'aaaa' },
]
documentService.getVersions.mockReturnValue(of({ id: 3, versions } as any))
component.newVersionLabel = ' Imported '
const versionsEmitSpy = jest.spyOn(component.versionsUpdated, 'emit')
const selectedEmitSpy = jest.spyOn(component.versionSelected, 'emit')
@@ -412,18 +410,11 @@ describe('DocumentVersionDropdownComponent', () => {
'Imported'
)
expect(documentService.updateVersionLabel).not.toHaveBeenCalled()
expect(documentService.getVersions).not.toHaveBeenCalled()
expect(versionsEmitSpy).not.toHaveBeenCalled()
expect(selectedEmitSpy).not.toHaveBeenCalled()
expect(documentService.getVersions).toHaveBeenCalledWith(3)
expect(versionsEmitSpy).toHaveBeenCalledWith(versions)
expect(selectedEmitSpy).toHaveBeenCalledWith(2)
expect(component.newVersionLabel).toEqual('')
expect(modal.close).toHaveBeenCalled()
jest.advanceTimersByTime(400)
expect(toastService.show).toHaveBeenCalledWith(
expect.objectContaining({
content: 'Existing document queued to be added as a version.',
actionName: 'View task',
})
)
jest.useRealTimers()
expect(toastService.showSuccess).toHaveBeenCalled()
})
})
@@ -28,7 +28,6 @@ import { DocumentVersionInfo } from 'src/app/data/document'
import { IfPermissionsDirective } from 'src/app/directives/if-permissions.directive'
import { CustomDatePipe } from 'src/app/pipes/custom-date.pipe'
import { DocumentService } from 'src/app/services/rest/document.service'
import { TaskNotificationService } from 'src/app/services/task-notification.service'
import { ToastService } from 'src/app/services/toast.service'
import {
UploadState,
@@ -78,7 +77,6 @@ export class DocumentVersionDropdownComponent
private readonly toastService = inject(ToastService)
private readonly websocketStatusService = inject(WebsocketStatusService)
private readonly modalService = inject(NgbModal)
private readonly taskNotifications = inject(TaskNotificationService)
private readonly destroy$ = new Subject<void>()
private readonly documentChange$ = new Subject<void>()
@@ -307,19 +305,23 @@ export class DocumentVersionDropdownComponent
versionLabel
)
.pipe(
switchMap(() => this.documentsService.getVersions(this.documentId)),
first(),
finalize(() => dialog.buttonsEnabled.set(true)),
takeUntil(this.destroy$),
takeUntil(this.documentChange$)
)
.subscribe({
next: ({ task_id }) => {
next: (document) => {
if (document?.versions?.length) {
this.versionsUpdated.emit(document.versions)
// The API returns versions newest first
this.versionSelected.emit(document.versions[0].id)
}
this.newVersionLabel = ''
modal.close()
this.taskNotifications.showQueued(
task_id,
$localize`Existing document queued to be added as a version.`,
{ content: $localize`Existing document added as a version.` }
this.toastService.showSuccess(
$localize`Existing document added as a version.`
)
},
error: (error) => {
@@ -14,8 +14,7 @@ import { Correspondent } from 'src/app/data/correspondent'
import { CustomField, CustomFieldDataType } from 'src/app/data/custom-field'
import { DocumentType } from 'src/app/data/document-type'
import { FILTER_TITLE } from 'src/app/data/filter-rule-type'
import { PaperlessTaskStatus } from 'src/app/data/paperless-task'
import { Results, SelectionData } from 'src/app/data/results'
import { Results } from 'src/app/data/results'
import { StoragePath } from 'src/app/data/storage-path'
import { Tag } from 'src/app/data/tag'
import { WorkflowTriggerType } from 'src/app/data/workflow-trigger'
@@ -29,7 +28,10 @@ import {
import { CorrespondentService } from 'src/app/services/rest/correspondent.service'
import { CustomFieldsService } from 'src/app/services/rest/custom-fields.service'
import { DocumentTypeService } from 'src/app/services/rest/document-type.service'
import { DocumentService } from 'src/app/services/rest/document.service'
import {
DocumentService,
SelectionData,
} from 'src/app/services/rest/document.service'
import { GroupService } from 'src/app/services/rest/group.service'
import { ShareLinkBundleService } from 'src/app/services/rest/share-link-bundle.service'
import { StoragePathService } from 'src/app/services/rest/storage-path.service'
@@ -37,7 +39,6 @@ import { TagService } from 'src/app/services/rest/tag.service'
import { UserService } from 'src/app/services/rest/user.service'
import { WorkflowService } from 'src/app/services/rest/workflow.service'
import { SettingsService } from 'src/app/services/settings.service'
import { TaskNotificationService } from 'src/app/services/task-notification.service'
import { ToastService } from 'src/app/services/toast.service'
import { TrashService } from 'src/app/services/trash.service'
import { environment } from 'src/environments/environment'
@@ -718,6 +719,7 @@ describe('BulkEditorComponent', () => {
'Workflow will run on the selected documents in the background.'
)
expect(toast.mock.calls.at(-1)[0].actionName).toBeUndefined()
expect(documentListViewService.selected.size).toBe(0)
httpTestingController.match(
`${environment.apiBaseUrl}documents/?page=1&page_size=50&ordering=-created&truncate_content=true&include_selection_data=true`
) // list reload
@@ -1265,15 +1267,14 @@ describe('BulkEditorComponent', () => {
let req = httpTestingController.expectOne(
`${environment.apiBaseUrl}documents/delete/`
)
const trackSpy = jest
.spyOn(trashService, 'trackMovedToTrash')
const notifySpy = jest
.spyOn(trashService, 'notifyMovedToTrash')
.mockImplementation(() => {})
req.flush({ result: 'OK', task_id: 'delete-1' })
req.flush(true)
expect(req.request.body).toEqual({
documents: [3, 4],
from_webui: true,
})
expect(trackSpy).toHaveBeenCalledWith('delete-1', 2, [3, 4])
expect(notifySpy).toHaveBeenCalledWith(expect.any(String), [3, 4])
httpTestingController.match(
`${environment.apiBaseUrl}documents/?page=1&page_size=50&ordering=-created&truncate_content=true&include_selection_data=true`
) // list reload
@@ -1290,9 +1291,9 @@ describe('BulkEditorComponent', () => {
req = httpTestingController.expectOne(
`${environment.apiBaseUrl}documents/delete/`
)
req.flush({ result: 'OK', task_id: 'delete-2' })
req.flush(true)
// ids aren't known when all matching documents are selected, so no undo
expect(trackSpy).toHaveBeenLastCalledWith('delete-2', 2, null)
expect(notifySpy).toHaveBeenLastCalledWith(expect.any(String), null)
httpTestingController.match(
`${environment.apiBaseUrl}documents/?page=1&page_size=50&ordering=-created&truncate_content=true&include_selection_data=true`
) // list reload
@@ -1372,11 +1373,10 @@ describe('BulkEditorComponent', () => {
let req = httpTestingController.expectOne(
`${environment.apiBaseUrl}documents/reprocess/`
)
req.flush({ result: 'OK', task_id: 'reprocess-1' })
req.flush(true)
expect(req.request.body).toEqual({
documents: [3, 4],
remote_ocr: false,
from_webui: true,
})
httpTestingController.match(
`${environment.apiBaseUrl}documents/?page=1&page_size=50&ordering=-created&truncate_content=true&include_selection_data=true`
@@ -1387,7 +1387,6 @@ describe('BulkEditorComponent', () => {
})
it('should support rotate', () => {
const toast = jest.spyOn(toastService, 'show')
let modal: NgbModalRef
modalService.activeInstances.subscribe((m) => (modal = m[0]))
jest.spyOn(permissionsService, 'currentUserCan').mockReturnValue(true)
@@ -1408,20 +1407,11 @@ describe('BulkEditorComponent', () => {
let req = httpTestingController.expectOne(
`${environment.apiBaseUrl}documents/rotate/`
)
req.flush({ result: 'OK', task_id: 'rotate-1' })
TestBed.inject(TaskNotificationService).complete({
task_id: 'rotate-1',
operation: 'rotate',
status: PaperlessTaskStatus.Success,
})
expect(toast.mock.calls.at(-1)[0].content).toBe(
"Bulk operation 'Rotate documents' submitted for processing."
)
req.flush(true)
expect(req.request.body).toEqual({
documents: [3, 4],
degrees: 90,
source_mode: 'latest_version',
from_webui: true,
})
httpTestingController.match(
`${environment.apiBaseUrl}documents/?page=1&page_size=50&ordering=-created&truncate_content=true&include_selection_data=true`
@@ -1459,11 +1449,10 @@ describe('BulkEditorComponent', () => {
let req = httpTestingController.expectOne(
`${environment.apiBaseUrl}documents/merge/`
)
req.flush({ result: 'OK', task_id: 'merge-1' })
req.flush(true)
expect(req.request.body).toEqual({
documents: [3, 4],
metadata_document_id: 3,
from_webui: true,
})
httpTestingController.match(
`${environment.apiBaseUrl}documents/?page=1&page_size=50&ordering=-created&truncate_content=true&include_selection_data=true`
@@ -1478,12 +1467,11 @@ describe('BulkEditorComponent', () => {
req = httpTestingController.expectOne(
`${environment.apiBaseUrl}documents/merge/`
)
req.flush({ result: 'OK', task_id: 'merge-2' })
req.flush(true)
expect(req.request.body).toEqual({
documents: [3, 4],
metadata_document_id: 3,
delete_originals: true,
from_webui: true,
})
httpTestingController.match(
`${environment.apiBaseUrl}documents/?page=1&page_size=50&ordering=-created&truncate_content=true&include_selection_data=true`
@@ -1491,6 +1479,7 @@ describe('BulkEditorComponent', () => {
httpTestingController.match(
`${environment.apiBaseUrl}documents/?page=1&page_size=100000&fields=id`
) // listAllFilteredIds
expect(documentListViewService.selected.size).toEqual(0)
// Test with archiveFallback enabled
modal.componentInstance.deleteOriginals.set(false)
@@ -1499,12 +1488,11 @@ describe('BulkEditorComponent', () => {
req = httpTestingController.expectOne(
`${environment.apiBaseUrl}documents/merge/`
)
req.flush({ result: 'OK', task_id: 'merge-3' })
req.flush(true)
expect(req.request.body).toEqual({
documents: [3, 4],
metadata_document_id: 3,
archive_fallback: true,
from_webui: true,
})
httpTestingController.match(
`${environment.apiBaseUrl}documents/?page=1&page_size=50&ordering=-created&truncate_content=true&include_selection_data=true`
@@ -1512,10 +1500,10 @@ describe('BulkEditorComponent', () => {
httpTestingController.match(
`${environment.apiBaseUrl}documents/?page=1&page_size=100000&fields=id`
) // listAllFilteredIds
expect(documentListViewService.selected.size).toEqual(0)
})
it('should support merging documents as versions', () => {
jest.useFakeTimers()
let modal: NgbModalRef
modalService.activeInstances.subscribe((m) => (modal = m[0]))
jest.spyOn(permissionsService, 'currentUserCan').mockReturnValue(true)
@@ -1543,8 +1531,8 @@ describe('BulkEditorComponent', () => {
.mockReturnValue(true)
const mergeAsVersionsSpy = jest
.spyOn(documentService, 'mergeDocumentsAsVersions')
.mockReturnValue(of({ result: 'OK', task_id: 'versions-1' }))
const toastSpy = jest.spyOn(toastService, 'show')
.mockReturnValue(of(true))
const toastSuccessSpy = jest.spyOn(toastService, 'showSuccess')
fixture.detectChanges()
component.mergeSelectedAsVersions()
@@ -1559,21 +1547,10 @@ describe('BulkEditorComponent', () => {
httpTestingController.match(
`${environment.apiBaseUrl}documents/?page=1&page_size=100000&fields=id`
)
jest.advanceTimersByTime(400)
expect(toastSpy).toHaveBeenCalledWith(
expect.objectContaining({
content: "Bulk operation 'Merge as versions' queued.",
})
expect(documentListViewService.selected.size).toEqual(0)
expect(toastSuccessSpy).toHaveBeenCalledWith(
'Documents merged as versions.'
)
TestBed.inject(TaskNotificationService).complete({
task_id: 'versions-1',
operation: 'merge_as_versions',
status: PaperlessTaskStatus.Success,
})
expect(toastSpy.mock.calls.at(-1)[0].content).toBe(
"Bulk operation 'Merge as versions' completed."
)
jest.useRealTimers()
})
it('should not report success when merging documents as versions fails', () => {
@@ -1664,22 +1641,12 @@ describe('BulkEditorComponent', () => {
)
})
it.each<[boolean, number]>([
[true, 2],
[false, 2],
[true, 1],
])(
'names queued work and gates the task link (permission %s, documents %s)',
(canViewTasks, count) => {
jest.useFakeTimers()
it.each([true, false])(
'shows queued feedback and gates the task link on permission (%s)',
(canViewTasks) => {
jest
.spyOn(permissionsService, 'currentUserCan')
.mockReturnValue(canViewTasks)
jest
.spyOn(documentListViewService, 'selected', 'get')
.mockReturnValue(
new Set(Array.from({ length: count }, (_, index) => index + 3))
)
const queued = jest.spyOn(toastService, 'show')
let modal: NgbModalRef
modalService.activeInstances.subscribe((instances) => {
@@ -1697,11 +1664,8 @@ describe('BulkEditorComponent', () => {
)
req.flush({ result: 'OK', task_id: 'queued-operation' })
expect(close).toHaveBeenCalledTimes(1)
expect(queued).not.toHaveBeenCalled()
jest.advanceTimersByTime(400)
const notification = queued.mock.calls.at(-1)[0]
const prefix = count > 1 ? 'Bulk operation' : 'Operation'
expect(notification.content).toBe(`${prefix} \'Set permissions\' queued.`)
expect(notification.content).toBe('Bulk operation queued.')
if (canViewTasks) {
expect(notification.actionName).toBe('View task')
notification.action()
@@ -1713,16 +1677,8 @@ describe('BulkEditorComponent', () => {
expect(notification.actionName).toBeUndefined()
}
expect(reload).not.toHaveBeenCalled()
expect(documentListViewService.selected.size).toBe(0)
expect(documentListViewService.allSelected).toBe(false)
TestBed.inject(TaskNotificationService).complete({
task_id: 'queued-operation',
operation: 'set_permissions',
status: PaperlessTaskStatus.Success,
})
const completed = queued.mock.calls.at(-1)[0]
expect(completed.content).toBe(`${prefix} \'Set permissions\' completed.`)
expect(completed.action).toBeUndefined()
jest.useRealTimers()
}
)
@@ -1803,6 +1759,7 @@ describe('BulkEditorComponent', () => {
{ id: 2, name: 'Tag 2' },
],
count: 2,
all: [1, 2],
}
const modalInstance = {
@@ -1845,6 +1802,7 @@ describe('BulkEditorComponent', () => {
{ id: 2, name: 'Correspondent 2' },
],
count: 2,
all: [1, 2],
}
const modalInstance = {
@@ -1893,6 +1851,7 @@ describe('BulkEditorComponent', () => {
{ id: 2, name: 'Document Type 2' },
],
count: 2,
all: [1, 2],
}
const modalInstance = {
@@ -1938,6 +1897,7 @@ describe('BulkEditorComponent', () => {
{ id: 2, name: 'Storage Path 2' },
],
count: 2,
all: [1, 2],
}
const modalInstance = {
@@ -1991,6 +1951,7 @@ describe('BulkEditorComponent', () => {
},
],
count: 2,
all: [1, 2],
}
const modalInstance = {
@@ -19,18 +19,25 @@ import {
NgbModalRef,
} from '@ng-bootstrap/ng-bootstrap'
import { saveAs } from 'file-saver'
import { first, map, Observable, Subject, switchMap, takeUntil } from 'rxjs'
import {
first,
map,
Observable,
Subject,
switchMap,
takeUntil,
tap,
} from 'rxjs'
import { BootstrapIconComponent } from 'src/app/components/common/bootstrap-icon/bootstrap-icon.component'
import { ConfirmDialogComponent } from 'src/app/components/common/confirm-dialog/confirm-dialog.component'
import { BulkOperationResponse } from 'src/app/data/bulk-operation'
import { CustomField } from 'src/app/data/custom-field'
import { MatchingModel } from 'src/app/data/matching-model'
import { SelectionDataItem } from 'src/app/data/results'
import { SETTINGS_KEYS } from 'src/app/data/ui-settings'
import { Workflow } from 'src/app/data/workflow'
import { WorkflowTriggerType } from 'src/app/data/workflow-trigger'
import { IfPermissionsDirective } from 'src/app/directives/if-permissions.directive'
import { DocumentListViewService } from 'src/app/services/document-list-view.service'
import { OpenDocumentsService } from 'src/app/services/open-documents.service'
import {
PermissionAction,
PermissionsService,
@@ -45,18 +52,14 @@ import {
DocumentService,
MergeDocumentsRequest,
} from 'src/app/services/rest/document.service'
import { SavedViewService } from 'src/app/services/rest/saved-view.service'
import { ShareLinkBundleService } from 'src/app/services/rest/share-link-bundle.service'
import { WorkflowService } from 'src/app/services/rest/workflow.service'
import { StoragePathService } from 'src/app/services/rest/storage-path.service'
import { TagService } from 'src/app/services/rest/tag.service'
import { WorkflowService } from 'src/app/services/rest/workflow.service'
import { SettingsService } from 'src/app/services/settings.service'
import { TaskNotificationService } from 'src/app/services/task-notification.service'
import { ToastService } from 'src/app/services/toast.service'
import { TrashService } from 'src/app/services/trash.service'
import {
documentOperationHasFollowUpTasks,
getDocumentOperationLabel,
} from 'src/app/utils/document-operation'
import { flattenTags } from 'src/app/utils/flatten-tags'
import { queryParamsFromFilterRules } from 'src/app/utils/query-params'
import { MergeAsVersionsConfirmDialogComponent } from '../../common/confirm-dialog/merge-as-versions-confirm-dialog/merge-as-versions-confirm-dialog.component'
@@ -81,6 +84,7 @@ import { PermissionsDialogComponent } from '../../common/permissions-dialog/perm
import { ShareLinkBundleDialogComponent } from '../../common/share-link-bundle-dialog/share-link-bundle-dialog.component'
import { ComponentWithPermissions } from '../../with-permissions/with-permissions.component'
import { CustomFieldsBulkEditDialogComponent } from './custom-fields-bulk-edit-dialog/custom-fields-bulk-edit-dialog.component'
import { WorkflowTriggerType } from 'src/app/data/workflow-trigger'
@Component({
selector: 'pngx-bulk-editor',
@@ -106,12 +110,13 @@ export class BulkEditorComponent
private documentService = inject(DocumentService)
private trashService = inject(TrashService)
private modalService = inject(NgbModal)
private openDocumentService = inject(OpenDocumentsService)
private settings = inject(SettingsService)
private toastService = inject(ToastService)
private taskNotifications = inject(TaskNotificationService)
private storagePathService = inject(StoragePathService)
private customFieldService = inject(CustomFieldsService)
public readonly permissionService = inject(PermissionsService)
private savedViewService = inject(SavedViewService)
private readonly shareLinkBundleService = inject(ShareLinkBundleService)
private readonly workflowService = inject(WorkflowService)
private readonly router = inject(Router)
@@ -329,6 +334,7 @@ export class BulkEditorComponent
.subscribe({
next: () => {
modal.close()
this.list.selectNone()
this.toastService.showInfo(
$localize`Workflow will run on the selected documents in the background.`
)
@@ -351,61 +357,70 @@ export class BulkEditorComponent
.bulkEdit(overrideSelection ?? this.getSelectionQuery(), method, args)
.pipe(first())
.subscribe({
next: ({ task_id }) =>
this.handleOperationQueued(modal, task_id, method),
next: ({ task_id }) => this.handleOperationQueued(modal, task_id),
error: (error) => this.handleOperationError(modal, error),
})
}
private handleOperationQueued(
modal: NgbModalRef,
taskId: string,
operation: string,
onQueued?: (taskId: string) => void
) {
const bulk = this.getSelectionSize() > 1
private handleOperationQueued(modal: NgbModalRef, taskId: string) {
modal?.close()
if (onQueued) {
onQueued(taskId)
} else {
const label = getDocumentOperationLabel(operation)
const submittedMessage = bulk
? $localize`Bulk operation '${label}' submitted for processing.`
: $localize`Operation '${label}' submitted for processing.`
const completedMessage = bulk
? $localize`Bulk operation '${label}' completed.`
: $localize`Operation '${label}' completed.`
this.taskNotifications.showQueued(
taskId,
bulk
? $localize`Bulk operation '${label}' queued.`
: $localize`Operation '${label}' queued.`,
{
content: documentOperationHasFollowUpTasks(operation)
? submittedMessage
: completedMessage,
}
)
}
this.list.selectNone()
const canViewTasks = this.permissionService.currentUserCan(
PermissionAction.View,
PermissionType.PaperlessTask
)
this.toastService.show({
content: $localize`Bulk operation queued.`,
delay: 5000,
...(canViewTasks
? {
actionName: $localize`View task`,
action: () =>
this.router.navigate(['/tasks'], {
queryParams: { task_id: taskId },
}),
}
: {}),
})
}
private executeDocumentAction(
modal: NgbModalRef,
operation: string,
request: Observable<BulkOperationResponse>,
onQueued?: (taskId: string) => void
request: Observable<any>,
options: { clearSelection?: boolean; successMessage?: string } = {}
) {
if (modal) {
modal.componentInstance.buttonsEnabled.set(false)
}
request.pipe(first()).subscribe({
next: ({ task_id }) => {
this.handleOperationQueued(modal, task_id, operation, onQueued)
next: () => {
this.handleOperationSuccess(modal, options.clearSelection ?? false)
if (options.successMessage) {
this.toastService.showSuccess(options.successMessage)
}
},
error: (error) => this.handleOperationError(modal, error),
})
}
private handleOperationSuccess(
modal: NgbModalRef,
clearSelection: boolean = false
) {
if (clearSelection) {
this.list.selected.clear()
}
this.list.reload()
this.list.reduceSelectionToFilter()
this.list.selected.forEach((id) => {
this.openDocumentService.refreshDocument(id)
})
this.savedViewService.maybeRefreshDocumentCounts()
if (modal) {
modal.close()
}
}
private handleOperationError(modal: NgbModalRef, error: any) {
if (modal) {
modal.componentInstance.buttonsEnabled.set(true)
@@ -973,9 +988,16 @@ export class BulkEditorComponent
const count = this.getSelectionSize()
this.executeDocumentAction(
modal,
'delete',
this.documentService.deleteDocuments(this.getSelectionQuery()),
(taskId) => this.trashService.trackMovedToTrash(taskId, count, documents)
this.documentService
.deleteDocuments(this.getSelectionQuery())
.pipe(
tap(() =>
this.trashService.notifyMovedToTrash(
$localize`Moved ${count} document(s) to the trash`,
documents
)
)
)
)
}
@@ -1016,7 +1038,6 @@ export class BulkEditorComponent
modal.componentInstance.buttonsEnabled.set(false)
this.executeDocumentAction(
modal,
'reprocess',
this.documentService.reprocessDocuments(
this.getSelectionQuery(),
modal.componentInstance.remoteOcr
@@ -1057,7 +1078,6 @@ export class BulkEditorComponent
rotateDialog.buttonsEnabled.set(false)
this.executeDocumentAction(
modal,
'rotate',
this.documentService.rotateDocuments(
this.getSelectionQuery(),
rotateDialog.degrees
@@ -1091,8 +1111,11 @@ export class BulkEditorComponent
mergeDialog.buttonsEnabled.set(false)
this.executeDocumentAction(
modal,
'merge',
this.documentService.mergeDocuments(mergeDialog.documentIDs(), args)
this.documentService.mergeDocuments(mergeDialog.documentIDs(), args),
{ clearSelection: !!args.delete_originals }
)
this.toastService.showInfo(
$localize`Merged document will be queued for consumption.`
)
})
}
@@ -1114,11 +1137,14 @@ export class BulkEditorComponent
.subscribe(() => {
this.executeDocumentAction(
modal,
'merge_as_versions',
this.documentService.mergeDocumentsAsVersions(
mergeDialog.documentIDs(),
mergeDialog.rootDocumentID()
)
),
{
clearSelection: true,
successMessage: $localize`Documents merged as versions.`,
}
)
})
}
@@ -1143,7 +1169,7 @@ export class BulkEditorComponent
dialog.selection = this.getSelectionQuery()
dialog.selectionCount = this.getSelectionSize()
dialog.succeeded.subscribe(({ task_id }) =>
this.handleOperationQueued(null, task_id, 'modify_custom_fields')
this.handleOperationQueued(null, task_id)
)
dialog.failed.subscribe((error) => {
this.toastService.showError(
@@ -240,7 +240,7 @@ describe('OpenDocumentsService', () => {
expect(refreshed.tags).toEqual([4])
})
it('keeps documents open after a network error during refresh', () => {
it('should handle error on refresh documents', () => {
subscriptions.push(
openDocumentsService.openDocument(documents[1]).subscribe()
)
@@ -250,63 +250,7 @@ describe('OpenDocumentsService', () => {
)
expect(req.request.method).toEqual('GET')
req.error(new ErrorEvent('timeout'))
expect(openDocumentsService.getOpenDocument(documents[1].id)).toBe(
documents[1]
)
})
it.each([500, 403])(
'keeps documents open after HTTP %s during refresh',
(status) => {
const doc = { ...documents[0] }
openDocumentsService.openDocument(doc).subscribe()
openDocumentsService.refreshDocument(doc.id)
httpTestingController
.expectOne(
`${environment.apiBaseUrl}documents/${doc.id}/?full_perms=true`
)
.flush({}, { status, statusText: 'Error' })
expect(openDocumentsService.getOpenDocument(doc.id)).toBe(doc)
}
)
it('preserves unsaved edits when a refreshed document no longer exists', () => {
const doc = { ...documents[0], title: 'Unsaved title' }
openDocumentsService.openDocument(doc).subscribe()
openDocumentsService.setDirty(doc, true, { title: doc.title })
openDocumentsService.refreshDocument(doc.id)
httpTestingController
.expectOne(
`${environment.apiBaseUrl}documents/${doc.id}/?full_perms=true`
)
.flush({}, { status: 404, statusText: 'Not Found' })
expect(openDocumentsService.getOpenDocument(doc.id)).toBe(doc)
expect(openDocumentsService.isDirty(doc)).toBe(true)
expect(
JSON.parse(sessionStorage.getItem(OPEN_DOCUMENT_SERVICE.DOCUMENTS))[0]
.title
).toBe('Unsaved title')
})
it('removes only the missing document when tab positions change during refresh', () => {
const first = { ...documents[0] }
const second = { ...documents[1] }
openDocumentsService.openDocument(first).subscribe()
openDocumentsService.openDocument(second).subscribe()
openDocumentsService.refreshDocument(first.id)
openDocumentsService.refreshDocument(second.id)
const firstRequest = httpTestingController.expectOne(
`${environment.apiBaseUrl}documents/${first.id}/?full_perms=true`
)
const secondRequest = httpTestingController.expectOne(
`${environment.apiBaseUrl}documents/${second.id}/?full_perms=true`
)
openDocumentsService.closeDocument(second).subscribe()
firstRequest.flush({}, { status: 404, statusText: 'Not Found' })
expect(openDocumentsService.getOpenDocuments()).toHaveLength(0)
openDocumentsService.openDocument(first).subscribe()
secondRequest.flush({}, { status: 404, statusText: 'Not Found' })
expect(openDocumentsService.getOpenDocuments()).toEqual([first])
})
it('should log error on sessionStorage save', () => {
@@ -1,4 +1,3 @@
import { HttpErrorResponse } from '@angular/common/http'
import { Injectable, inject, signal } from '@angular/core'
import { NgbModal } from '@ng-bootstrap/ng-bootstrap'
import { Observable, Subject, of } from 'rxjs'
@@ -62,12 +61,7 @@ export class OpenDocumentsService {
Object.assign(openDoc, doc, unsavedEdits)
this.save()
},
error: (error: HttpErrorResponse) => {
if (error.status !== 404) return
const index = this.openDocuments.findIndex((doc) => doc.id === id)
if (index === -1 || this.isDirty(this.openDocuments[index])) return
error: () => {
this.openDocuments.splice(index, 1)
this.save()
},
@@ -290,7 +290,6 @@ describe(`DocumentService`, () => {
expect(req.request.method).toEqual('POST')
expect(req.request.body).toEqual({
documents: ids,
from_webui: true,
})
})
@@ -304,7 +303,6 @@ describe(`DocumentService`, () => {
expect(req.request.body).toEqual({
documents: ids,
remote_ocr: false,
from_webui: true,
})
})
@@ -319,7 +317,6 @@ describe(`DocumentService`, () => {
expect(req.request.body).toEqual({
documents: ids,
remote_ocr: true,
from_webui: true,
})
})
@@ -334,7 +331,6 @@ describe(`DocumentService`, () => {
documents: ids,
degrees: 90,
source_mode: 'latest_version',
from_webui: true,
})
})
@@ -350,7 +346,6 @@ describe(`DocumentService`, () => {
documents: ids,
metadata_document_id: 1,
delete_originals: true,
from_webui: true,
})
})
@@ -364,7 +359,6 @@ describe(`DocumentService`, () => {
expect(req.request.body).toEqual({
documents: ids,
root_document_id: 2,
from_webui: true,
})
})
@@ -380,7 +374,6 @@ describe(`DocumentService`, () => {
documents: ids,
root_document_id: 2,
version_label: 'Imported',
from_webui: true,
})
})
@@ -395,7 +388,6 @@ describe(`DocumentService`, () => {
expect(req.request.body).toEqual({
documents: ids,
operations: [{ page: 1, rotate: 90, doc: 0 }],
from_webui: true,
})
})
@@ -411,7 +403,6 @@ describe(`DocumentService`, () => {
documents: ids,
password: 'secret',
update_document: true,
from_webui: true,
})
})
@@ -351,24 +351,19 @@ export class DocumentService extends AbstractPaperlessService<Document> {
}
deleteDocuments(selection: DocumentSelectionQuery) {
return this.http.post<BulkOperationResponse>(
this.getResourceUrl(null, 'delete'),
{ ...selection, from_webui: true }
)
return this.http.post(this.getResourceUrl(null, 'delete'), {
...selection,
})
}
reprocessDocuments(
selection: DocumentSelectionQuery,
remoteOcr: boolean = false
) {
return this.http.post<BulkOperationResponse>(
this.getResourceUrl(null, 'reprocess'),
{
...selection,
remote_ocr: remoteOcr,
from_webui: true,
}
)
return this.http.post(this.getResourceUrl(null, 'reprocess'), {
...selection,
remote_ocr: remoteOcr,
})
}
rotateDocuments(
@@ -376,26 +371,18 @@ export class DocumentService extends AbstractPaperlessService<Document> {
degrees: number,
sourceMode: BulkEditSourceMode = BulkEditSourceMode.LATEST_VERSION
) {
return this.http.post<BulkOperationResponse>(
this.getResourceUrl(null, 'rotate'),
{
...selection,
degrees,
source_mode: sourceMode,
from_webui: true,
}
)
return this.http.post(this.getResourceUrl(null, 'rotate'), {
...selection,
degrees,
source_mode: sourceMode,
})
}
mergeDocuments(ids: number[], request: MergeDocumentsRequest = {}) {
return this.http.post<BulkOperationResponse>(
this.getResourceUrl(null, 'merge'),
{
documents: ids,
...request,
from_webui: true,
}
)
return this.http.post(this.getResourceUrl(null, 'merge'), {
documents: ids,
...request,
})
}
mergeDocumentsAsVersions(
@@ -403,40 +390,28 @@ export class DocumentService extends AbstractPaperlessService<Document> {
rootDocumentId: number,
versionLabel?: string
) {
return this.http.post<BulkOperationResponse>(
this.getResourceUrl(null, 'merge_as_versions'),
{
documents: ids,
root_document_id: rootDocumentId,
...(versionLabel ? { version_label: versionLabel } : {}),
from_webui: true,
}
)
return this.http.post(this.getResourceUrl(null, 'merge_as_versions'), {
documents: ids,
root_document_id: rootDocumentId,
...(versionLabel ? { version_label: versionLabel } : {}),
})
}
editPdfDocuments(ids: number[], request: EditPdfDocumentsRequest) {
return this.http.post<BulkOperationResponse>(
this.getResourceUrl(null, 'edit_pdf'),
{
documents: ids,
...request,
from_webui: true,
}
)
return this.http.post(this.getResourceUrl(null, 'edit_pdf'), {
documents: ids,
...request,
})
}
removePasswordDocuments(
ids: number[],
request: RemovePasswordDocumentsRequest
) {
return this.http.post<BulkOperationResponse>(
this.getResourceUrl(null, 'remove_password'),
{
documents: ids,
...request,
from_webui: true,
}
)
return this.http.post(this.getResourceUrl(null, 'remove_password'), {
documents: ids,
...request,
})
}
getSelectionData(
@@ -1,186 +0,0 @@
import { TestBed } from '@angular/core/testing'
import { Router } from '@angular/router'
import { BulkOperationStatus } from '../data/bulk-operation'
import { PaperlessTaskStatus } from '../data/paperless-task'
import { PermissionsService } from './permissions.service'
import { TaskNotificationService } from './task-notification.service'
import { Toast, ToastLevel, ToastService } from './toast.service'
describe('TaskNotificationService', () => {
let service: TaskNotificationService
let toastService: ToastService
let popups: Toast[]
let notifications: Toast[]
const navigate = jest.fn()
const currentUserCan = jest.fn()
const result = (
status = PaperlessTaskStatus.Success
): BulkOperationStatus => ({
task_id: 'task-1',
operation: 'run_workflow',
status,
})
beforeEach(() => {
jest.useFakeTimers()
currentUserCan.mockReturnValue(true)
TestBed.configureTestingModule({
providers: [
{ provide: Router, useValue: { navigate } },
{ provide: PermissionsService, useValue: { currentUserCan } },
],
})
service = TestBed.inject(TaskNotificationService)
toastService = TestBed.inject(ToastService)
popups = []
notifications = []
toastService.showToast.subscribe((toast) => popups.push(toast))
toastService
.getToasts()
.subscribe((toasts) => (notifications = [...toasts]))
})
afterEach(() => {
jest.clearAllTimers()
jest.useRealTimers()
})
it('replaces a queued notification with its result, removing the task link on success', () => {
service.showQueued('task-1', 'Workflow queued.', {
content: 'Workflow completed.',
})
jest.advanceTimersByTime(399)
expect(popups).toHaveLength(0)
jest.advanceTimersByTime(1)
expect(popups[0].content).toBe('Workflow queued.')
expect(popups[0].actionName).toBe('View task')
popups[0].action()
expect(navigate).toHaveBeenCalledWith(['/tasks'], {
queryParams: { task_id: 'task-1' },
})
service.complete(result())
expect(notifications).toHaveLength(1)
expect(popups[1]).toEqual(
expect.objectContaining({
id: popups[0].id,
content: 'Workflow completed.',
level: ToastLevel.Success,
})
)
expect(popups[1].action).toBeUndefined()
expect(popups[1].actionName).toBeUndefined()
})
it('shows only the result when a task completes during the grace period', () => {
service.showQueued('task-1', 'Queued')
jest.advanceTimersByTime(100)
service.complete(result())
jest.advanceTimersByTime(1000)
expect(popups.map((toast) => toast.content)).toEqual([
"Operation 'Run workflow' completed.",
])
})
it.each([
'reprocess',
'rotate',
'merge',
'split',
'delete_pages',
'edit_pdf',
'remove_password',
])('reports %s as submitted rather than completed', (operation) => {
service.showQueued('task-1', 'Queued')
service.complete({ ...result(), operation })
expect(notifications[0].content).toContain('submitted for processing.')
expect(notifications[0].content).not.toContain('completed')
expect(notifications[0].action).toBeUndefined()
})
it('does not show a stale queued state when completion beats the HTTP response', () => {
service.complete(result())
service.showQueued('task-1', 'Workflow queued.', {
content: 'Workflow completed.',
})
jest.advanceTimersByTime(1000)
expect(popups.every((toast) => toast.level === ToastLevel.Success)).toBe(
true
)
expect(notifications).toHaveLength(1)
expect(notifications[0].content).toBe('Workflow completed.')
expect(notifications[0].action).toBeUndefined()
})
it.each([PaperlessTaskStatus.Failure, PaperlessTaskStatus.Revoked])(
'replaces the queued state with an error for %s',
(status) => {
service.showQueued('task-1', 'Queued', {
content: 'Success',
actionName: 'Undo',
})
jest.advanceTimersByTime(400)
service.complete({ ...result(status), error: 'Task failed' })
expect(notifications).toHaveLength(1)
expect(notifications[0]).toEqual(
expect.objectContaining({
level: ToastLevel.Error,
error: 'Task failed',
actionName: 'View task',
})
)
}
)
it('ignores nonterminal and duplicate outcomes', () => {
service.showQueued('task-1', 'Queued')
service.complete(result(PaperlessTaskStatus.Started))
expect(popups).toHaveLength(0)
service.complete(result())
service.complete(result())
expect(popups).toHaveLength(1)
})
it('keeps concurrent task notifications separate', () => {
service.showQueued('task-1', 'First queued')
service.showQueued('task-2', 'Second queued')
jest.advanceTimersByTime(400)
service.complete(result())
expect(notifications).toHaveLength(2)
expect(notifications.map((toast) => toast.content)).toEqual([
'Second queued',
"Operation 'Run workflow' completed.",
])
})
it('omits task links without task viewing permission', () => {
currentUserCan.mockReturnValue(false)
service.showQueued('task-1', 'Queued')
jest.advanceTimersByTime(400)
service.complete(result())
expect(popups.every((toast) => !toast.action && !toast.actionName)).toBe(
true
)
})
it('identifies concurrent operations even when completion arrives before their queue responses', () => {
service.complete({
task_id: 'tags',
operation: 'modify_tags',
status: PaperlessTaskStatus.Success,
})
service.complete({
task_id: 'correspondent',
operation: 'set_correspondent',
status: PaperlessTaskStatus.Success,
})
expect(notifications.map((toast) => toast.content)).toEqual([
"Operation 'Set correspondent' completed.",
"Operation 'Update tags' completed.",
])
})
it('keeps the generic fallback for unknown operation names', () => {
service.complete({ ...result(), operation: 'future_operation' })
expect(notifications[0].content).toBe('Operation completed.')
})
})
@@ -1,128 +0,0 @@
import { inject, Injectable, OnDestroy } from '@angular/core'
import { Router } from '@angular/router'
import { BulkOperationStatus } from '../data/bulk-operation'
import { PaperlessTaskStatus } from '../data/paperless-task'
import {
documentOperationHasFollowUpTasks,
getDocumentOperationLabel,
} from '../utils/document-operation'
import {
PermissionAction,
PermissionsService,
PermissionType,
} from './permissions.service'
import { Toast, ToastLevel, ToastService } from './toast.service'
interface PendingNotification {
success: Partial<Toast>
timer: ReturnType<typeof setTimeout>
}
@Injectable({ providedIn: 'root' })
export class TaskNotificationService implements OnDestroy {
private readonly toastService = inject(ToastService)
private readonly router = inject(Router)
private readonly permissionsService = inject(PermissionsService)
private readonly pending = new Map<string, PendingNotification>()
private readonly completed = new Map<string, BulkOperationStatus>()
ngOnDestroy() {
this.pending.forEach(({ timer }) => clearTimeout(timer))
}
showQueued(taskId: string, content: string, success: Partial<Toast> = {}) {
const completed = this.completed.get(taskId)
if (completed) {
this.showResult(completed, success)
return
}
if (this.pending.has(taskId)) return
this.pending.set(taskId, {
success,
// short delay in case the task finishes super fast
timer: setTimeout(() => this.show(taskId, { content, delay: 5000 }), 400),
})
}
complete(result: BulkOperationStatus) {
if (
![
PaperlessTaskStatus.Success,
PaperlessTaskStatus.Failure,
PaperlessTaskStatus.Revoked,
].includes(result.status) ||
this.completed.has(result.task_id)
)
return
this.completed.set(result.task_id, result)
// Retain recent outcomes in case task finishes quickly
if (this.completed.size > 100) {
this.completed.delete(this.completed.keys().next().value)
}
const pending = this.pending.get(result.task_id)
if (pending) {
clearTimeout(pending.timer)
this.pending.delete(result.task_id)
}
// Trash needs the caller's document IDs to offer Undo on success.
if (
pending ||
result.operation !== 'delete' ||
result.status !== PaperlessTaskStatus.Success
) {
this.showResult(result, pending?.success)
}
}
private showResult(
result: BulkOperationStatus,
success: Partial<Toast> = {}
) {
const succeeded = result.status === PaperlessTaskStatus.Success
const operation = getDocumentOperationLabel(result.operation)
let content: string
if (succeeded) {
let operationText = $localize`Operation`
if (operation) operationText += ` '${operation}'`
const resultText = documentOperationHasFollowUpTasks(result.operation)
? $localize`submitted for processing`
: $localize`completed`
content = `${operationText} ${resultText}.`
} else if (result.status === PaperlessTaskStatus.Revoked) {
content = operation
? $localize`Operation '${operation}' revoked.`
: $localize`Operation revoked.`
} else {
content = operation
? $localize`Operation '${operation}' failed. Some changes may already have been applied.`
: $localize`Operation failed. Some changes may already have been applied.`
}
this.show(result.task_id, {
content,
delay: succeeded ? 5000 : 10000,
level: succeeded ? ToastLevel.Success : ToastLevel.Error,
...(succeeded ? success : { error: result.error }),
})
}
private show(taskId: string, toast: Toast) {
const canViewTasks = this.permissionsService.currentUserCan(
PermissionAction.View,
PermissionType.PaperlessTask
)
this.toastService.show({
id: `task-${taskId}`,
...(canViewTasks && toast.level !== ToastLevel.Success
? {
actionName: $localize`View task`,
action: () =>
this.router.navigate(['/tasks'], {
queryParams: { task_id: taskId },
}),
}
: {}),
...toast,
})
}
}
@@ -38,29 +38,6 @@ describe('ToastService', () => {
})
})
it('updates a notification by ID without duplicating or moving it', () => {
let notifications: Toast[] = []
toastService
.getToasts()
.subscribe((toasts) => (notifications = [...toasts]))
toastService.show({ id: 'task-1', content: 'Queued', delay: 5000 })
toastService.show({
id: 'other',
content: 'Another notification',
delay: 5000,
})
toastService.show({
id: 'task-1',
content: 'Completed',
delay: 5000,
level: ToastLevel.Success,
})
expect(notifications.map((toast) => toast.content)).toEqual([
'Another notification',
'Completed',
])
})
it('parses error string to object on show', () => {
const toast = {
title: 'Title',
+1 -8
View File
@@ -75,14 +75,7 @@ export class ToastService {
if (persistent) {
// the dropdown is open, so it's seen as it arrives
toast.read = !!this._suppressPopupToasts
const index = this.toasts.findIndex(
(existing) => existing.id === toast.id
)
if (index === -1) {
this.toasts.unshift(toast)
} else {
this.toasts[index] = toast
}
this.toasts.unshift(toast)
this.toastsSubject.next(this.toasts)
}
// Popups are suppressed while the dropdown is open
+1 -51
View File
@@ -5,10 +5,8 @@ import {
HttpTestingController,
} from '@angular/common/http/testing'
import { environment } from 'src/environments/environment'
import { PaperlessTaskStatus } from '../data/paperless-task'
import { DocumentListViewService } from './document-list-view.service'
import { TaskNotificationService } from './task-notification.service'
import { Toast, ToastLevel, ToastService } from './toast.service'
import { Toast, ToastService } from './toast.service'
import { TrashService } from './trash.service'
describe('TrashService', () => {
@@ -107,52 +105,4 @@ describe('TrashService', () => {
expect(toast.content).toEqual('Moved 500 document(s) to the trash')
expect(toast.action).toBeUndefined()
})
it('offers Undo only after the matching delete task succeeds', () => {
let toasts: Toast[] = []
toastService.getToasts().subscribe((t) => (toasts = [...t]))
service.trackMovedToTrash('delete-1', 2, [1, 2])
expect(toasts).toHaveLength(0)
const notifications = TestBed.inject(TaskNotificationService)
notifications.complete({
task_id: 'other-delete',
operation: 'delete',
status: PaperlessTaskStatus.Success,
})
expect(toasts).toHaveLength(0)
notifications.complete({
task_id: 'delete-1',
operation: 'delete',
status: PaperlessTaskStatus.Success,
})
expect(toasts).toHaveLength(1)
expect(toasts[0].actionName).toBe('Undo')
toasts[0].action()
const req = httpTestingController.expectOne(
`${environment.apiBaseUrl}trash/`
)
expect(req.request.body).toEqual({ action: 'restore', documents: [1, 2] })
})
it('handles completion before the queue response and does not offer Undo on failure', () => {
let toasts: Toast[] = []
toastService.getToasts().subscribe((t) => (toasts = [...t]))
const notifications = TestBed.inject(TaskNotificationService)
notifications.complete({
task_id: 'delete-1',
operation: 'delete',
status: PaperlessTaskStatus.Success,
})
service.trackMovedToTrash('delete-1', 1, [1])
expect(toasts[0].actionName).toBe('Undo')
service.trackMovedToTrash('delete-2', 1, [2])
notifications.complete({
task_id: 'delete-2',
operation: 'delete',
status: PaperlessTaskStatus.Failure,
})
expect(toasts).toHaveLength(2)
expect(toasts[0].level).toBe(ToastLevel.Error)
expect(toasts[0].actionName).not.toBe('Undo')
})
})
+5 -32
View File
@@ -5,8 +5,7 @@ import { environment } from 'src/environments/environment'
import { Document } from '../data/document'
import { Results } from '../data/results'
import { DocumentListViewService } from './document-list-view.service'
import { TaskNotificationService } from './task-notification.service'
import { Toast, ToastLevel, ToastService } from './toast.service'
import { ToastLevel, ToastService } from './toast.service'
@Injectable({
providedIn: 'root',
@@ -15,24 +14,6 @@ export class TrashService {
private http = inject(HttpClient)
private toastService = inject(ToastService)
private documentListViewService = inject(DocumentListViewService)
private taskNotifications = inject(TaskNotificationService)
public trackMovedToTrash(
taskId: string,
count: number,
documents?: number[]
) {
this.taskNotifications.showQueued(
taskId,
count > 1
? $localize`Bulk move to trash queued.`
: $localize`Move to trash queued.`,
this.movedToTrashToast(
$localize`Moved ${count} document(s) to the trash`,
documents
)
)
}
public getTrash(page: number = 1): Observable<Results<Document>> {
const httpParams = new HttpParams().set('page', page.toString())
@@ -62,22 +43,14 @@ export class TrashService {
* Shows a toast with an Undo action when possible
*/
public notifyMovedToTrash(content: string, documents?: number[]) {
this.toastService.show(this.movedToTrashToast(content, documents))
}
private movedToTrashToast(content: string, documents?: number[]): Toast {
const canUndo = documents?.length > 0
return {
this.toastService.show({
content,
delay: 10000,
level: ToastLevel.Success,
...(canUndo
? {
actionName: $localize`Undo`,
action: () => this.undoMoveToTrash(documents),
}
: {}),
}
actionName: canUndo ? $localize`Undo` : undefined,
action: canUndo ? () => this.undoMoveToTrash(documents) : undefined,
})
}
private undoMoveToTrash(documents: number[]) {
@@ -1,11 +0,0 @@
import { getDocumentOperationLabel } from './document-operation'
describe('Document Operation Utils', () => {
it('should retrieve the document operation label correctly for an existing operation, null otherwise', () => {
const result = getDocumentOperationLabel('modify_custom_fields')
expect(result).toBe('Update custom fields')
const resultNull = getDocumentOperationLabel('non_existing_operation')
expect(resultNull).toBeNull()
})
})
@@ -1,38 +0,0 @@
const OPERATION_LABELS: Record<string, string> = {
set_correspondent: $localize`Set correspondent`,
set_document_type: $localize`Set document type`,
set_storage_path: $localize`Set storage path`,
add_tag: $localize`Add tag`,
remove_tag: $localize`Remove tag`,
modify_tags: $localize`Update tags`,
modify_custom_fields: $localize`Update custom fields`,
set_permissions: $localize`Set permissions`,
delete: $localize`Move to trash`,
reprocess: $localize`Document reprocessing`,
run_workflow: $localize`Run workflow`,
rotate: $localize`Rotate documents`,
merge: $localize`Merge documents`,
merge_as_versions: $localize`Merge as versions`,
split: $localize`Split documents`,
delete_pages: $localize`Delete pages`,
edit_pdf: $localize`Edit PDF`,
remove_password: $localize`Remove password`,
}
export function getDocumentOperationLabel(operation?: string): string | null {
const label = operation ? OPERATION_LABELS[operation] : null
return label ?? null
}
// These operations queue processing in separate tasks
export function documentOperationHasFollowUpTasks(operation?: string): boolean {
return [
'reprocess',
'rotate',
'merge',
'split',
'delete_pages',
'edit_pdf',
'remove_password',
].includes(operation ?? '')
}
+1 -1
View File
@@ -8,7 +8,7 @@ export const environment = {
apiVersion: '10', // match src/paperless/settings.py
appTitle: DEFAULT_APP_TITLE,
tag: 'prod',
version: '3.5.0',
version: '3.4.2',
webSocketHost: window.location.host,
webSocketProtocol: window.location.protocol == 'https:' ? 'wss:' : 'ws:',
webSocketBaseUrl: base_url.pathname + 'ws/',
File diff suppressed because it is too large. Load diff
File diff suppressed because it is too large. Load diff
File diff suppressed because it is too large. Load diff
File diff suppressed because it is too large. Load diff
File diff suppressed because it is too large. Load diff
File diff suppressed because it is too large. Load diff
File diff suppressed because it is too large. Load diff
File diff suppressed because it is too large. Load diff
File diff suppressed because it is too large. Load diff
File diff suppressed because it is too large. Load diff
File diff suppressed because it is too large. Load diff
File diff suppressed because it is too large. Load diff
File diff suppressed because it is too large. Load diff
File diff suppressed because it is too large. Load diff
File diff suppressed because it is too large. Load diff
File diff suppressed because it is too large. Load diff
File diff suppressed because it is too large. Load diff
File diff suppressed because it is too large. Load diff
File diff suppressed because it is too large. Load diff
File diff suppressed because it is too large. Load diff
File diff suppressed because it is too large. Load diff
File diff suppressed because it is too large. Load diff
File diff suppressed because it is too large. Load diff
File diff suppressed because it is too large. Load diff
File diff suppressed because it is too large. Load diff
File diff suppressed because it is too large. Load diff
File diff suppressed because it is too large. Load diff
File diff suppressed because it is too large. Load diff
File diff suppressed because it is too large. Load diff
File diff suppressed because it is too large. Load diff
File diff suppressed because it is too large. Load diff
File diff suppressed because it is too large. Load diff
File diff suppressed because it is too large. Load diff
File diff suppressed because it is too large. Load diff
File diff suppressed because it is too large. Load diff
File diff suppressed because it is too large. Load diff
File diff suppressed because it is too large. Load diff
File diff suppressed because it is too large. Load diff
File diff suppressed because it is too large. Load diff
File diff suppressed because it is too large. Load diff
File diff suppressed because it is too large. Load diff
File diff suppressed because it is too large. Load diff
File diff suppressed because it is too large. Load diff
File diff suppressed because it is too large. Load diff
File diff suppressed because it is too large. Load diff
File diff suppressed because it is too large. Load diff
File diff suppressed because it is too large. Load diff
-6
View File
@@ -153,12 +153,6 @@ def fake_progress_manager(
return FakeProgressManager
@pytest.fixture(scope="session")
def shared_samples_dir() -> Path:
"""Directory of sample files used by more than one app's tests."""
return samples.SHARED_SAMPLES_DIR
@pytest.fixture(scope="session")
def simple_digital_pdf_file() -> Path:
"""One-page PDF with a text layer."""
+2 -29
View File
@@ -427,7 +427,6 @@ def delete(doc_ids: list[int]) -> Literal["OK"]:
"Detected a possible incompatible database column. See https://docs.paperless-ngx.com/troubleshooting/#convert-uuid-field",
)
logger.error(f"Error deleting documents: {e!s}")
raise
return "OK"
@@ -475,9 +474,9 @@ def run_workflow(
"""
Manually run a single workflow against the given documents.
Called from the tracked bulk operation task.
Done asynchronously via a Celery task.
"""
run_workflows_for_documents(
run_workflows_for_documents.delay(
document_ids=doc_ids,
workflow_id=workflow,
filter_documents=filter_documents,
@@ -539,7 +538,6 @@ def rotate(
pair = _resolve_root_and_source_doc(doc, source_mode=source_mode)
docs_by_root_id.setdefault(pair.root_doc.id, pair)
failed_ids = []
for pair in docs_by_root_id.values():
if pair.source_doc.mime_type != "application/pdf":
logger.warning(
@@ -574,14 +572,8 @@ def rotate(
f"Queued new rotated version for document {pair.root_doc.id} by {degrees} degrees",
)
except Exception as e:
failed_ids.append(pair.root_doc.id)
logger.exception(f"Error rotating document {pair.root_doc.id}: {e}")
if failed_ids:
raise RuntimeError(
f"Rotation failed for document(s): {failed_ids}. Check logs for details.",
)
return "OK"
@@ -692,22 +684,6 @@ def merge(
return "OK"
def validate_merge_as_versions_state(
documents: list[Document],
root_document_id: int,
) -> None:
if any(document.root_document_id is not None for document in documents):
raise ValueError("Only top-level documents can be merged as versions.")
source_ids = [
document.pk for document in documents if document.pk != root_document_id
]
if Document.global_objects.filter(root_document_id__in=source_ids).exists():
raise ValueError(
"Documents with existing versions cannot be merged into another document.",
)
def merge_as_versions(
doc_ids: list[int],
*,
@@ -725,7 +701,6 @@ def merge_as_versions(
)
documents_by_id = {document.id: document for document in documents}
validate_merge_as_versions_state(documents, root_document_id)
source_ids = [doc_id for doc_id in doc_ids if doc_id != root_document_id]
root_document = documents_by_id[root_document_id]
next_version_index = (
@@ -889,7 +864,6 @@ def split(
except Exception as e:
logger.exception(f"Error splitting document {doc.id}: {e}")
raise
return "OK"
@@ -935,7 +909,6 @@ def delete_pages(
)
except Exception as e:
logger.exception(f"Error deleting pages from document {pair.root_doc.id}: {e}")
raise
return "OK"
+2 -65
View File
@@ -1,4 +1,4 @@
"""Queued document bulk operations."""
"""Queued document metadata edits."""
from typing import Any
@@ -12,7 +12,6 @@ from rest_framework.exceptions import PermissionDenied
from documents import bulk_edit
from documents.models import CustomFieldInstance
from documents.models import Document
from documents.models import PaperlessTask
from documents.permissions import has_document_operation_permissions
from documents.tasks import bulk_update_documents
@@ -26,35 +25,7 @@ MODIFIED_FIELDS = {
"modify_custom_fields": "custom_fields",
"set_permissions": None,
}
LEGACY_MODIFIED_FIELDS = {"delete": "deleted_at", "reprocess": "checksum"}
QUEUED_OPERATIONS = frozenset(
(
*MODIFIED_FIELDS,
"delete",
"reprocess",
"run_workflow",
"rotate",
"merge",
"merge_as_versions",
"split",
"delete_pages",
"edit_pdf",
"remove_password",
),
)
USER_OPERATIONS = frozenset(
{
"split",
"merge",
"rotate",
"delete_pages",
"edit_pdf",
"remove_password",
"merge_as_versions",
},
)
# merge_as_versions doesn't queue any consume tasks
TRIGGER_SOURCE_OPERATIONS = USER_OPERATIONS - {"merge_as_versions"}
QUEUED_OPERATIONS = frozenset(MODIFIED_FIELDS)
def snapshot_field(doc_ids: list[int], field: str) -> dict[int, Any]:
@@ -95,7 +66,6 @@ def execute_bulk_operation(
document_ids: list[int],
parameters: dict,
user_id: int,
trigger_source: str = PaperlessTask.TriggerSource.API_UPLOAD,
) -> dict:
if operation not in QUEUED_OPERATIONS:
raise ValueError(f"Unsupported bulk operation: {operation}")
@@ -109,39 +79,6 @@ def execute_bulk_operation(
):
raise PermissionDenied("Insufficient permissions to execute bulk operation")
if operation == "run_workflow" and not user.has_perm("documents.view_workflow"):
raise PermissionDenied("Insufficient permissions to run this workflow")
if operation not in MODIFIED_FIELDS:
execution_parameters = parameters.copy()
if operation in USER_OPERATIONS:
execution_parameters["user"] = user
if operation in TRIGGER_SOURCE_OPERATIONS:
execution_parameters["trigger_source"] = trigger_source
modified_field = LEGACY_MODIFIED_FIELDS.get(operation)
with set_actor(user):
old_values = (
snapshot_field(document_ids, modified_field)
if settings.AUDIT_LOG_ENABLED and modified_field
else {}
)
method(document_ids, **execution_parameters)
if settings.AUDIT_LOG_ENABLED and modified_field:
from auditlog.models import LogEntry
new_values = snapshot_field(document_ids, modified_field)
for doc in Document.objects.filter(pk__in=document_ids):
LogEntry.objects.log_create(
instance=doc,
changes={
modified_field: [old_values[doc.pk], new_values[doc.pk]],
},
action=LogEntry.Action.UPDATE,
actor=user,
additional_data={"reason": f"Bulk edit: {operation}"},
)
return {"operation": operation, "document_count": len(document_ids)}
affected_ids: list[int] = []
modified_field = MODIFIED_FIELDS[operation]
# Keep metadata writes and their audit entries together. Follow-up processing
+3 -45
View File
@@ -3,6 +3,7 @@ from __future__ import annotations
import logging
import re
from fnmatch import fnmatch
from fnmatch import translate as fnmatch_translate
from typing import TYPE_CHECKING
from rest_framework import serializers
@@ -547,48 +548,6 @@ def existing_document_matches_workflow(
return (True, None)
def _glob_to_portable_regex(pattern: str) -> str:
"""
Translate an fnmatch-style glob into an anchored regex which is accepted by
Python, PostgreSQL (ARE), and MariaDB (PCRE).
fnmatch.translate() can't be used, as its output (`(?s:...)\\Z`) uses syntax
PostgreSQL rejects. Supports `*`, `?`, `[seq]` and `[!seq]`.
"""
out: list[str] = []
i = 0
n = len(pattern)
while i < n:
c = pattern[i]
i += 1
if c == "*":
out.append(".*")
elif c == "?":
out.append(".")
elif c == "[":
j = i
if j < n and pattern[j] == "!":
j += 1
if j < n and pattern[j] == "]":
j += 1
while j < n and pattern[j] != "]":
j += 1
if j >= n:
# No closing bracket, treat as a literal
out.append(r"\[")
else:
body = pattern[i:j]
i = j + 1
negate = body.startswith("!")
if negate:
body = body[1:]
body = "".join(f"\\{ch}" if ch in "\\[^" else ch for ch in body)
out.append(f"[{'^' if negate else ''}{body}]")
else:
out.append(re.escape(c))
return f"^{''.join(out)}$"
def prefilter_documents_by_workflowtrigger(
documents: QuerySet[Document],
trigger: WorkflowTrigger,
@@ -669,9 +628,8 @@ def prefilter_documents_by_workflowtrigger(
documents = documents.annotate(**annotations).filter(custom_field_q)
if trigger.filter_filename:
documents = documents.filter(
original_filename__iregex=_glob_to_portable_regex(trigger.filter_filename),
)
regex = fnmatch_translate(trigger.filter_filename).lstrip("^").rstrip("$")
documents = documents.filter(original_filename__iregex=regex)
return documents
-4
View File
@@ -138,10 +138,6 @@ class Tag(MatchingModel):
return TagHierarchy(cls.objects.values_list("pk", "tn_parent_id"))
@property
def parent_pk(self) -> int | None:
return self.tn_parent_id
def get_parent(self) -> "Tag | None":
return self.tn_parent
-93
View File
@@ -4,21 +4,13 @@ import logging
import mimetypes
import shutil
import subprocess
import tempfile
from pathlib import Path
from typing import TYPE_CHECKING
from typing import Final
from django.conf import settings
from documents.loggers import LoggingMixin
from documents.utils import copy_file_with_basic_stats
from documents.utils import run_subprocess
from paperless.parsers.registry import get_parser_registry
if TYPE_CHECKING:
import datetime
logger = logging.getLogger("paperless.parsing")
@@ -44,17 +36,6 @@ def get_default_file_extension(mime_type: str) -> str:
return ext if ext else ""
def is_file_ext_supported(ext: str) -> bool:
"""
Returns True if the file extension is supported, False otherwise
TODO: Investigate why this really exists, why not use mimetype
"""
if ext:
return ext.lower() in get_supported_file_extensions()
else:
return False
def get_supported_file_extensions() -> set[str]:
extensions = set()
for parser_class in get_parser_registry().all_parsers():
@@ -276,77 +257,3 @@ def make_thumbnail_from_pdf(in_path: Path, temp_dir: Path, logging_group=None) -
class ParseError(Exception):
pass
class DocumentParser(LoggingMixin):
"""
Subclass this to make your own parser. Have a look at
`paperless_tesseract.parsers` for inspiration.
"""
logging_name = "paperless.parsing"
def __init__(self, logging_group, progress_callback=None) -> None:
super().__init__()
self.renew_logging_group()
self.logging_group = logging_group
self.settings = self.get_settings()
settings.SCRATCH_DIR.mkdir(parents=True, exist_ok=True)
self.tempdir = Path(
tempfile.mkdtemp(prefix="paperless-", dir=settings.SCRATCH_DIR),
)
self.archive_path = None
self.text = None
self.date: datetime.datetime | None = None
self.progress_callback = progress_callback
def progress(self, current_progress, max_progress) -> None:
if self.progress_callback:
self.progress_callback(current_progress, max_progress)
def get_settings(self): # pragma: no cover
"""
A parser must implement this
"""
raise NotImplementedError
def read_file_handle_unicode_errors(self, filepath: Path) -> str:
"""
Helper utility for reading from a file, and handling a problem with its
unicode, falling back to ignoring the error to remove the invalid bytes
"""
try:
text = filepath.read_text(encoding="utf-8")
except UnicodeDecodeError as e:
self.log.warning(f"Unicode error during text reading, continuing: {e}")
text = filepath.read_bytes().decode("utf-8", errors="replace")
return text
def extract_metadata(self, document_path, mime_type):
return []
def get_page_count(self, document_path, mime_type) -> None:
return None
def parse(self, document_path, mime_type, file_name=None):
raise NotImplementedError
def get_archive_path(self):
return self.archive_path
def get_thumbnail(self, document_path, mime_type, file_name=None):
"""
Returns the path to a file we can use as a thumbnail for this document.
"""
raise NotImplementedError
def get_text(self):
return self.text
def get_date(self) -> datetime.datetime | None:
return self.date
def cleanup(self) -> None:
self.log.debug(f"Deleting directory {self.tempdir}")
shutil.rmtree(self.tempdir)
+5 -22
View File
@@ -49,8 +49,6 @@ class SanityCheckMessages:
self._document_error_pks: set[int] = set()
self._document_warning_pks: set[int] = set()
self._document_info_pks: set[int] = set()
self._document_error_issue_count: int = 0
self._document_warning_issue_count: int = 0
self.has_error: bool = False
self.has_warning: bool = False
self.has_info: bool = False
@@ -76,17 +74,17 @@ class SanityCheckMessages:
def error(self, doc_pk: int | None, message: str) -> None:
self._messages[doc_pk].append({"level": logging.ERROR, "message": message})
self.has_error = True
if doc_pk is not None:
self._document_error_issue_count += 1
if self._add_document_issue(doc_pk, self._document_error_pks):
self.document_error_count += 1
if doc_pk is not None and self._add_document_issue(
doc_pk,
self._document_error_pks,
):
self.document_error_count += 1
def warning(self, doc_pk: int | None, message: str) -> None:
self._messages[doc_pk].append({"level": logging.WARNING, "message": message})
self.has_warning = True
if doc_pk is not None:
self._document_warning_issue_count += 1
if self._add_document_issue(doc_pk, self._document_warning_pks):
self.document_warning_count += 1
else:
@@ -116,21 +114,6 @@ class SanityCheckMessages:
def __getitem__(self, item: int | None) -> list[MessageEntry]:
return self._messages[item]
# -- Summarize Helpers --------------------------------------------------
@property
def has_global_issues(self) -> bool:
return None in self._messages
@property
def total_issue_count(self) -> int:
"""Total number of error and warning messages across all documents and global."""
return (
self._document_error_issue_count
+ self._document_warning_issue_count
+ self.global_warning_count
)
# -- Logging output (used by Celery task path) --------------------------
def log_messages(self) -> None:
+13 -50
View File
@@ -536,43 +536,6 @@ class DocumentTypeSerializer(MatchingModelSerializer, OwnedObjectSerializer):
)
class DeprecatedColors:
COLOURS = (
(1, "#a6cee3"),
(2, "#1f78b4"),
(3, "#b2df8a"),
(4, "#33a02c"),
(5, "#fb9a99"),
(6, "#e31a1c"),
(7, "#fdbf6f"),
(8, "#ff7f00"),
(9, "#cab2d6"),
(10, "#6a3d9a"),
(11, "#b15928"),
(12, "#000000"),
(13, "#cccccc"),
)
@extend_schema_field(
serializers.ChoiceField(
choices=DeprecatedColors.COLOURS,
),
)
class ColorField(serializers.Field):
def to_internal_value(self, data):
for id, color in DeprecatedColors.COLOURS:
if id == data:
return color
raise serializers.ValidationError
def to_representation(self, value):
for id, color in DeprecatedColors.COLOURS:
if color == value:
return id
return 1
class TagSerializer(MatchingModelSerializer, OwnedObjectSerializer):
def get_text_color(self, obj) -> str:
try:
@@ -1688,7 +1651,6 @@ class MergeDocumentsSerializer(DocumentListSerializer, SourceModeValidationMixin
class MergeDocumentsAsVersionsSerializer(DocumentListSerializer):
root_document_id = serializers.IntegerField(required=True)
from_webui = serializers.BooleanField(required=False, default=False)
version_label = serializers.CharField(
required=False,
allow_blank=True,
@@ -1717,17 +1679,19 @@ class MergeDocumentsAsVersionsSerializer(DocumentListSerializer):
"root_document_id must be one of the selected documents.",
)
# Rechecked under row locks during execution in case document state changes.
selected_documents = list(
Document.objects.filter(id__in=documents).only("id", "root_document"),
)
try:
bulk_edit.validate_merge_as_versions_state(
selected_documents,
attrs["root_document_id"],
selected_documents = Document.objects.filter(id__in=documents)
if selected_documents.filter(root_document__isnull=False).exists():
raise serializers.ValidationError(
"Only top-level documents can be merged as versions.",
)
source_document_ids = set(documents) - {attrs["root_document_id"]}
if Document.global_objects.filter(
root_document_id__in=source_document_ids,
).exists():
raise serializers.ValidationError(
"Documents with existing versions cannot be merged into another document.",
)
except ValueError as error:
raise serializers.ValidationError(str(error)) from error
return attrs
@@ -1800,12 +1764,11 @@ class RemovePasswordDocumentsSerializer(
class DeleteDocumentsSerializer(DocumentSelectionSerializer):
from_webui = serializers.BooleanField(required=False, default=False)
pass
class ReprocessDocumentsSerializer(DocumentSelectionSerializer):
remote_ocr = serializers.BooleanField(required=False, default=False)
from_webui = serializers.BooleanField(required=False, default=False)
class BulkEditSerializer(
-2
View File
@@ -1206,8 +1206,6 @@ def before_task_publish_handler(
"document_count": len(task_kwargs["document_ids"]),
}
owner_id = task_kwargs["user_id"]
if task_kwargs["operation"] == "delete":
task_type = PaperlessTask.TaskType.BULK_DELETE
else:
input_data = _extract_input_data(task_type, task_kwargs)
owner_id = _extract_owner_id(task_type, task_kwargs)
+1 -8
View File
@@ -713,9 +713,8 @@ def run_workflows_for_documents(
logger.warning(
f"Manual workflow run requested for missing workflow {workflow_id}",
)
raise
return
failed_ids = []
for document in Document.objects.filter(
id__in=document_ids,
root_document__isnull=True,
@@ -733,16 +732,10 @@ def run_workflows_for_documents(
document=document,
)
except Exception:
failed_ids.append(document.pk)
logger.exception(
f"Error running workflow {workflow} on document {document.pk}",
)
if failed_ids:
raise RuntimeError(
f"Workflow {workflow_id} failed for document(s): {failed_ids}. Check logs for details.",
)
def update_document_parent_tags(tag: Tag, new_parent: Tag) -> None:
"""
+2 -14
View File
@@ -4,7 +4,6 @@ from typing import TYPE_CHECKING
from unittest import mock
import pytest
from asgiref.sync import async_to_sync
from rest_framework import status
from rest_framework.test import APIClient
from rest_framework.test import APITestCase
@@ -13,8 +12,6 @@ from paperless_testing.factories import UserFactory
from paperless_testing.permissions import grant_global
if TYPE_CHECKING:
from collections.abc import AsyncIterator
from django.contrib.auth.models import User
from pytest_mock import MockerFixture
@@ -58,11 +55,6 @@ class TestChatStreamingViewInputValidation(APITestCase):
reaches the Django request the middleware sees.
"""
chunks = [f"token{i} " for i in range(40)]
async def answer() -> AsyncIterator[str]:
for chunk in chunks:
yield chunk
with (
mock.patch(
"documents.views.AIConfig",
@@ -70,7 +62,7 @@ class TestChatStreamingViewInputValidation(APITestCase):
),
mock.patch(
"paperless_ai.chat.stream_chat_with_documents",
return_value=answer(),
return_value=iter(chunks),
),
):
resp = self.client.post(
@@ -82,11 +74,7 @@ class TestChatStreamingViewInputValidation(APITestCase):
assert resp.status_code == status.HTTP_200_OK
assert not resp.has_header("Content-Encoding")
async def read_parts() -> list[bytes]:
return [part async for part in resp.streaming_content]
assert async_to_sync(read_parts)() == [c.encode() for c in chunks]
assert list(resp.streaming_content) == [c.encode() for c in chunks]
def test_missing_question_is_rejected(self) -> None:
with mock.patch(
+9 -69
View File
@@ -698,10 +698,10 @@ class TestBulkEdit(DirectoriesMixin, TestCase):
def test_delete_documents_old_uuid_field(self, m) -> None:
m.side_effect = Exception("Data too long for column 'transaction_id' at row 1")
doc_ids = [self.doc1.id, self.doc2.id, self.doc3.id]
bulk_edit.delete(doc_ids)
with self.assertLogs(level="WARNING") as cm:
with self.assertRaisesRegex(Exception, "Data too long for column"):
bulk_edit.delete(doc_ids)
self.assertIn("possible incompatible database column", cm.output[0])
bulk_edit.delete(doc_ids)
self.assertIn("possible incompatible database column", cm.output[0])
class TestPDFActions(DirectoriesMixin, TestCase):
@@ -1038,9 +1038,8 @@ class TestPDFActions(DirectoriesMixin, TestCase):
mock_consume_file.assert_not_called()
@mock.patch("documents.bulk_edit.group")
@mock.patch("documents.tasks.consume_file.s")
def test_split(self, mock_consume_file, mock_group) -> None:
def test_split(self, mock_consume_file) -> None:
"""
GIVEN:
- Existing documents
@@ -1061,7 +1060,6 @@ class TestPDFActions(DirectoriesMixin, TestCase):
self.assertIsNone(mock_consume_file.call_args.kwargs["overrides"].asn)
self.assertEqual(result, "OK")
mock_group.return_value.delay.assert_called_once()
@mock.patch("documents.bulk_edit.group")
@mock.patch("documents.pdf_ops.build_pdfs")
@@ -1170,8 +1168,8 @@ class TestPDFActions(DirectoriesMixin, TestCase):
sig.apply_async.side_effect = Exception("boom")
mock_chord.return_value = sig
with self.assertRaisesMessage(Exception, "boom"):
bulk_edit.split(doc_ids, pages, delete_originals=True)
result = bulk_edit.split(doc_ids, pages, delete_originals=True)
self.assertEqual(result, "OK")
self.doc2.refresh_from_db()
self.assertEqual(self.doc2.archive_serial_number, 222)
@@ -1193,8 +1191,7 @@ class TestPDFActions(DirectoriesMixin, TestCase):
pages = [[1, 2], [3]]
with self.assertLogs("paperless.bulk_edit", level="ERROR") as cm:
with self.assertRaisesMessage(Exception, "Error building PDFs"):
bulk_edit.split(doc_ids, pages)
bulk_edit.split(doc_ids, pages)
error_str = cm.output[0]
expected_str = "Error splitting document 2"
self.assertIn(expected_str, error_str)
@@ -1243,68 +1240,12 @@ class TestPDFActions(DirectoriesMixin, TestCase):
doc_ids = [self.doc2.id, self.doc3.id]
with self.assertLogs("paperless.bulk_edit", level="ERROR") as cm:
with self.assertRaisesMessage(
RuntimeError,
f"Rotation failed for document(s): {doc_ids}",
):
bulk_edit.rotate(doc_ids, 90)
bulk_edit.rotate(doc_ids, 90)
error_str = cm.output[0]
expected_str = "Error rotating document"
self.assertIn(expected_str, error_str)
self.assertEqual(mock_rotate_pdf.call_count, 2)
mock_consume_delay.assert_not_called()
@mock.patch("documents.tasks.consume_file.apply_async")
@mock.patch("documents.pdf_ops.rotate_pdf")
def test_rotate_continues_after_preparation_failure(
self,
mock_rotate_pdf,
mock_consume,
) -> None:
def rotate(source, destination, degrees):
if source == self.doc2.source_path:
raise RuntimeError("Invalid PDF")
shutil.copy(source, destination)
mock_rotate_pdf.side_effect = rotate
with self.assertRaisesMessage(
RuntimeError,
f"Rotation failed for document(s): [{self.doc2.id}]",
):
bulk_edit.rotate([self.doc2.id, self.doc3.id], 90)
self.assertEqual(mock_rotate_pdf.call_count, 2)
mock_consume.assert_called_once()
self.assertEqual(
mock_consume.call_args.kwargs["kwargs"]["input_doc"].root_document_id,
self.doc3.id,
)
@mock.patch("documents.tasks.consume_file.apply_async")
@mock.patch("documents.pdf_ops.rotate_pdf")
def test_rotate_continues_after_queue_failure(
self,
mock_rotate_pdf,
mock_consume,
) -> None:
mock_rotate_pdf.side_effect = lambda source, destination, degrees: shutil.copy(
source,
destination,
)
mock_consume.side_effect = [RuntimeError("Broker unavailable"), None]
with self.assertRaisesMessage(
RuntimeError,
f"Rotation failed for document(s): [{self.doc2.id}]",
):
bulk_edit.rotate([self.doc2.id, self.doc3.id], 90)
self.assertEqual(mock_rotate_pdf.call_count, 2)
self.assertEqual(mock_consume.call_count, 2)
self.assertEqual(
mock_consume.call_args.kwargs["kwargs"]["input_doc"].root_document_id,
self.doc3.id,
)
@mock.patch("documents.tasks.consume_file.apply_async")
def test_rotate_non_pdf(
self,
@@ -1434,8 +1375,7 @@ class TestPDFActions(DirectoriesMixin, TestCase):
pages = [1, 3]
with self.assertLogs("paperless.bulk_edit", level="ERROR") as cm:
with self.assertRaisesMessage(Exception, "Error removing pages"):
bulk_edit.delete_pages(doc_ids, pages)
bulk_edit.delete_pages(doc_ids, pages)
error_str = cm.output[0]
expected_str = "Error deleting pages from document"
self.assertIn(expected_str, error_str)
+15 -170
View File
@@ -54,7 +54,6 @@ def test_request_only_queues(bulk_setup, endpoint, operation, parameters):
"document_ids": [doc.pk],
"parameters": parameters,
"user_id": user.pk,
"trigger_source": "web_ui",
}
assert enqueue.call_args.kwargs["headers"]["trigger_source"] == "web_ui"
@@ -103,122 +102,6 @@ def test_permissions_rechecked_on_worker(bulk_setup):
assert doc.correspondent_id is None
@pytest.mark.django_db
def test_document_action_runs_with_user_in_worker(bulk_setup):
user, _, doc, _ = bulk_setup
with mock.patch("documents.bulk_edit.rotate") as rotate:
result = execute_bulk_operation(
"rotate",
[doc.pk],
{"degrees": 90},
user.pk,
trigger_source="web_ui",
)
rotate.assert_called_once_with(
[doc.pk],
degrees=90,
user=user,
trigger_source="web_ui",
)
assert result == {"operation": "rotate", "document_count": 1}
@pytest.mark.django_db
def test_document_action_queue_contains_only_serializable_parameters(bulk_setup):
user, client, doc, _ = bulk_setup
with mock.patch("documents.views.execute_bulk_operation.apply_async") as enqueue:
enqueue.return_value.id = "rotate-task"
response = client.post(
"/api/documents/rotate/",
{"documents": [doc.pk], "degrees": 90, "from_webui": True},
format="json",
)
assert response.data == {"result": "OK", "task_id": "rotate-task"}
assert enqueue.call_args.kwargs["kwargs"] == {
"operation": "rotate",
"document_ids": [doc.pk],
"parameters": {"degrees": 90, "source_mode": "latest_version"},
"user_id": user.pk,
"trigger_source": "web_ui",
}
@pytest.mark.django_db
def test_reprocess_keeps_bulk_audit_entry(bulk_setup, settings):
user, _, doc, _ = bulk_setup
settings.AUDIT_LOG_ENABLED = True
LogEntry.objects.all().delete()
with mock.patch("documents.bulk_edit.reprocess"):
execute_bulk_operation("reprocess", [doc.pk], {}, user.pk)
entry = LogEntry.objects.get_for_object(doc).get()
assert entry.changes_dict["checksum"] == [doc.checksum, doc.checksum]
assert entry.actor_id == user.pk
@pytest.mark.django_db
def test_workflow_runs_in_tracked_task_instead_of_dispatching_another(bulk_setup):
user, _, doc, _ = bulk_setup
with mock.patch("documents.bulk_edit.run_workflows_for_documents") as run:
execute_bulk_operation(
"run_workflow",
[doc.pk],
{"workflow": 7, "filter_documents": False},
user.pk,
)
run.assert_called_once_with(
document_ids=[doc.pk],
workflow_id=7,
filter_documents=False,
)
@pytest.mark.django_db
def test_delete_failure_propagates_from_worker(bulk_setup):
user, _, doc, _ = bulk_setup
with mock.patch(
"documents.models.Document.delete",
side_effect=RuntimeError("delete failed"),
):
with pytest.raises(RuntimeError, match="delete failed"):
execute_bulk_operation("delete", [doc.pk], {}, user.pk)
assert Document.objects.filter(pk=doc.pk).exists()
@pytest.mark.django_db
@pytest.mark.parametrize(
"operation,parameters,preparation",
[
("rotate", {"degrees": 90}, "rotate_pdf"),
("split", {"pages": [[1]]}, "build_pdfs"),
("delete_pages", {"pages": [1]}, "remove_pages"),
],
)
def test_preparation_failure_propagates_from_worker(
bulk_setup,
operation,
parameters,
preparation,
):
user, _, doc, _ = bulk_setup
doc.mime_type = "application/pdf"
doc.save()
with (
mock.patch(
f"documents.bulk_edit.pdf_ops.{preparation}",
side_effect=RuntimeError("preparation failed"),
),
mock.patch("documents.bulk_edit.consume_file.apply_async") as consume,
mock.patch("documents.bulk_edit.consume_file.s") as consume_signature,
pytest.raises(
RuntimeError,
match="Rotation failed" if operation == "rotate" else "preparation failed",
),
):
execute_bulk_operation(operation, [doc.pk], parameters, user.pk)
consume.assert_not_called()
consume_signature.assert_not_called()
@pytest.mark.django_db
def test_queue_failure_does_not_change_documents(bulk_setup):
_, client, doc, correspondent = bulk_setup
@@ -279,22 +162,6 @@ def test_tracked_outcome_records_initiator_and_result(bulk_setup, failed):
assert task.result_data == {"document_count": 1}
@pytest.mark.django_db
def test_queued_delete_is_tracked_as_bulk_delete(bulk_setup):
user, _, doc, _ = bulk_setup
before_task_publish_handler(
headers={"task": execute_bulk_operation.name, "id": "delete-task"},
body=(
(),
{"operation": "delete", "document_ids": [doc.pk], "user_id": user.pk},
{},
),
)
task = PaperlessTask.objects.get(task_id="delete-task")
assert task.task_type == PaperlessTask.TaskType.BULK_DELETE
assert task.owner_id == user.pk
@pytest.mark.django_db
def test_large_filtered_selection_is_one_task(bulk_setup):
_, client, _, _ = bulk_setup
@@ -417,46 +284,24 @@ async def test_completion_socket_is_private(anyio_backend):
@pytest.mark.django_db
def test_legacy_trash_queues_without_deleting_in_request(bulk_setup):
user, client, doc, _ = bulk_setup
data = {"documents": [doc.pk], "method": "delete", "parameters": {}}
with mock.patch("documents.views.execute_bulk_operation.apply_async") as enqueue:
enqueue.return_value.id = "delete-task"
@pytest.mark.parametrize("endpoint", ["delete", "bulk_edit"])
def test_trash_remains_synchronous(bulk_setup, endpoint):
_, client, doc, _ = bulk_setup
data = {"documents": [doc.pk]}
if endpoint == "bulk_edit":
data.update(method="delete", parameters={})
with (
mock.patch("documents.views.execute_bulk_operation.apply_async") as enqueue,
mock.patch("documents.search.get_backend"),
):
response = client.post(
"/api/documents/bulk_edit/",
f"/api/documents/{endpoint}/",
data,
format="json",
HTTP_ACCEPT="application/json; version=9",
)
assert response.status_code == 200
assert response.data == {"result": "OK", "task_id": "delete-task"}
assert Document.objects.filter(pk=doc.pk).exists()
assert enqueue.call_args.kwargs["kwargs"] == {
"operation": "delete",
"document_ids": [doc.pk],
"parameters": {},
"user_id": user.pk,
"trigger_source": "api_upload",
}
@pytest.mark.django_db
def test_dedicated_trash_queues_without_deleting_in_request(bulk_setup):
user, client, doc, _ = bulk_setup
with mock.patch("documents.views.execute_bulk_operation.apply_async") as enqueue:
enqueue.return_value.id = "delete-task"
response = client.post(
"/api/documents/delete/",
{"documents": [doc.pk]},
format="json",
)
assert response.status_code == 200
assert response.data == {"result": "OK", "task_id": "delete-task"}
assert Document.objects.filter(pk=doc.pk).exists()
assert enqueue.call_args.kwargs["kwargs"] == {
"operation": "delete",
"document_ids": [doc.pk],
"parameters": {},
"user_id": user.pk,
"trigger_source": "api_upload",
}
assert response.data == {"result": "OK"}
assert not Document.objects.filter(pk=doc.pk).exists()
assert Document.deleted_objects.filter(pk=doc.pk).exists()
enqueue.assert_not_called()
@@ -1,62 +0,0 @@
import fnmatch
import re
import pytest
from documents.matching import _glob_to_portable_regex
@pytest.mark.parametrize(
("pattern", "expected"),
[
pytest.param("", r"^$", id="empty"),
pytest.param("abc", r"^abc$", id="literal"),
pytest.param("*", r"^.*$", id="star"),
pytest.param("?", r"^.$", id="question"),
pytest.param("*.pdf", r"^.*\.pdf$", id="star-escapes-dot"),
pytest.param("[abc]", r"^[abc]$", id="class"),
pytest.param("[a-z]", r"^[a-z]$", id="class-range"),
pytest.param("[!abc]", r"^[^abc]$", id="negated-class"),
pytest.param("[]a]", r"^[]a]$", id="leading-close-bracket-is-member"),
pytest.param("[!]a]", r"^[^]a]$", id="negated-leading-close-bracket"),
pytest.param("[a^b]", r"^[a\^b]$", id="caret-escaped-in-class"),
pytest.param("[a[b]", r"^[a\[b]$", id="open-bracket-escaped-in-class"),
pytest.param("[a\\b]", r"^[a\\b]$", id="backslash-escaped-in-class"),
pytest.param("[abc", r"^\[abc$", id="unclosed-class-literal"),
pytest.param("[", r"^\[$", id="lone-open-bracket"),
pytest.param("[!", r"^\[!$", id="unclosed-negation-literal"),
pytest.param("[]", r"^\[\]$", id="unclosed-with-only-close-bracket"),
pytest.param("a[0-9]?*", r"^a[0-9]..*$", id="mixed"),
],
)
def test_translation(pattern: str, expected: str) -> None:
"""
GIVEN:
- A glob pattern
WHEN:
- It is translated to a portable regex
THEN:
- The exact anchored regex is produced
"""
assert _glob_to_portable_regex(pattern) == expected
@pytest.mark.parametrize(
"pattern",
["*", "?", "*.pdf", "a?c", "[abc]x", "[!abc]x", "[]a]", "[!]a]", "[a-c]*", "[a"],
)
@pytest.mark.parametrize(
"name",
["", "a", "abc", "x", "]", "]x", "file.pdf", "ax", "bx", "dx", "[a"],
)
def test_agrees_with_fnmatch(pattern: str, name: str) -> None:
"""
GIVEN:
- A glob pattern and a name
WHEN:
- Matched via the portable regex and via stdlib fnmatch
THEN:
- Both agree
"""
regex = _glob_to_portable_regex(pattern)
assert bool(re.search(regex, name)) == fnmatch.fnmatchcase(name, pattern)
@@ -305,7 +305,8 @@ class TestExportImport(
self.assertEqual(Permission.objects.count(), num_permission_objects)
messages = check_sanity()
# everything is alright after the test
self.assertEqual(messages.total_issue_count, 0)
self.assertFalse(messages.has_error)
self.assertFalse(messages.has_warning)
def test_exporter_with_filename_format(self) -> None:
shutil.rmtree(Path(self.dirs.media_dir) / "documents")
@@ -8,7 +8,6 @@ from rest_framework import status
from rest_framework.test import APITestCase
from documents.bulk_edit import merge_as_versions
from documents.bulk_operations import execute_bulk_operation
from documents.models import Document
from documents.serialisers import MergeDocumentsAsVersionsSerializer
from paperless_testing.factories import UserFactory
@@ -35,7 +34,6 @@ class TestMergeDocumentsAsVersionsSerializer(TestCase):
{
"documents": [self.doc1.id, self.doc2.id, self.doc3.id],
"root_document_id": self.doc2.id,
"from_webui": False,
},
)
@@ -108,6 +106,7 @@ class TestMergeDocumentsAsVersionsSerializer(TestCase):
def test_rejects_selected_version(self) -> None:
version = Document.objects.create(
checksum="D",
title="D",
root_document=self.doc1,
version_index=1,
)
@@ -117,12 +116,54 @@ class TestMergeDocumentsAsVersionsSerializer(TestCase):
"root_document_id": self.doc2.id,
},
)
self.assertFalse(serializer.is_valid())
self.assertEqual(
serializer.errors["non_field_errors"][0],
"Only top-level documents can be merged as versions.",
)
def test_rejects_source_document_with_versions(self) -> None:
Document.objects.create(
checksum="D",
title="D",
root_document=self.doc1,
version_index=1,
)
serializer = MergeDocumentsAsVersionsSerializer(
data={
"documents": [self.doc1.id, self.doc2.id],
"root_document_id": self.doc2.id,
},
)
self.assertFalse(serializer.is_valid())
self.assertEqual(
serializer.errors["non_field_errors"][0],
"Documents with existing versions cannot be merged into another document.",
)
def test_rejects_source_document_with_trashed_versions(self) -> None:
version = Document.objects.create(
checksum="D",
title="D",
root_document=self.doc1,
version_index=1,
)
version.delete() # trashed, but still points at doc1
serializer = MergeDocumentsAsVersionsSerializer(
data={
"documents": [self.doc1.id, self.doc2.id],
"root_document_id": self.doc2.id,
},
)
self.assertFalse(serializer.is_valid())
self.assertEqual(
serializer.errors["non_field_errors"][0],
"Documents with existing versions cannot be merged into another document.",
)
def test_allows_root_document_with_versions(self) -> None:
Document.objects.create(
checksum="D",
@@ -141,95 +182,6 @@ class TestMergeDocumentsAsVersionsSerializer(TestCase):
class TestMergeDocumentsAsVersions(TestCase):
def setUp(self) -> None:
self.doc1 = Document.objects.create(checksum="test-A", title="A")
self.doc2 = Document.objects.create(checksum="test-B", title="B")
def test_rejects_selected_version(self) -> None:
version = Document.objects.create(
checksum="D",
title="D",
root_document=self.doc1,
version_index=1,
)
with self.assertRaisesMessage(
ValueError,
"Only top-level documents can be merged as versions.",
):
merge_as_versions(
[version.id, self.doc2.id],
root_document_id=self.doc2.id,
)
def test_rejects_source_document_with_versions(self) -> None:
Document.objects.create(
checksum="D",
title="D",
root_document=self.doc1,
version_index=1,
)
with self.assertRaisesMessage(
ValueError,
"Documents with existing versions cannot be merged into another document.",
):
merge_as_versions(
[self.doc1.id, self.doc2.id],
root_document_id=self.doc2.id,
)
def test_rejects_source_document_with_trashed_versions(self) -> None:
version = Document.objects.create(
checksum="D",
title="D",
root_document=self.doc1,
version_index=1,
)
version.delete() # trashed, but still points at doc1
with self.assertRaisesMessage(
ValueError,
"Documents with existing versions cannot be merged into another document.",
):
merge_as_versions(
[self.doc1.id, self.doc2.id],
root_document_id=self.doc2.id,
)
@mock.patch("documents.bulk_edit.DocumentsStatusManager")
@mock.patch("documents.bulk_edit.bulk_update_documents.apply_async")
@mock.patch("documents.search.get_backend")
def test_rejects_source_that_gained_versions_after_validation(
self,
_get_backend_mock,
_bulk_update_mock,
_status_manager_mock,
) -> None:
destination = Document.objects.create(checksum="destination")
serializer = MergeDocumentsAsVersionsSerializer(
data={
"documents": [self.doc1.id, destination.id],
"root_document_id": destination.id,
},
)
self.assertTrue(serializer.is_valid(), serializer.errors)
merge_as_versions(
[self.doc1.id, self.doc2.id],
root_document_id=self.doc1.id,
)
with self.assertRaisesMessage(
ValueError,
"Documents with existing versions cannot be merged into another document.",
):
merge_as_versions(
serializer.validated_data["documents"],
root_document_id=serializer.validated_data["root_document_id"],
)
self.doc1.refresh_from_db()
self.doc2.refresh_from_db()
self.assertIsNone(self.doc1.root_document_id)
self.assertEqual(self.doc2.root_document_id, self.doc1.id)
@mock.patch("documents.bulk_edit.DocumentsStatusManager")
@mock.patch("documents.bulk_edit.bulk_update_documents.apply_async")
@mock.patch("documents.search.get_backend")
@@ -396,17 +348,6 @@ class TestMergeDocumentsAsVersionsAPI(APITestCase):
owner=self.user,
)
self.client.force_authenticate(user=self.user)
enqueue = mock.patch(
"documents.views.execute_bulk_operation.apply_async",
side_effect=lambda *, kwargs, headers: self.run_operation(kwargs),
)
enqueue.start()
self.addCleanup(enqueue.stop)
@staticmethod
def run_operation(kwargs):
execute_bulk_operation(**kwargs)
return mock.Mock(id="test-bulk-operation")
@mock.patch("documents.views.bulk_edit.merge_as_versions")
def test_merges_documents_as_versions(self, merge_mock) -> None:
@@ -426,10 +367,7 @@ class TestMergeDocumentsAsVersionsAPI(APITestCase):
)
self.assertEqual(response.status_code, status.HTTP_200_OK)
self.assertEqual(
response.data,
{"result": "OK", "task_id": "test-bulk-operation"},
)
self.assertEqual(response.data, {"result": "OK"})
merge_mock.assert_called_once_with(
[self.doc1.id, self.doc2.id],
root_document_id=self.doc2.id,
@@ -502,36 +440,27 @@ class TestMergeDocumentsAsVersionsAPI(APITestCase):
self.assertEqual(response.status_code, status.HTTP_400_BAD_REQUEST)
merge_mock.assert_not_called()
@mock.patch("documents.views.execute_bulk_operation.apply_async")
def test_rejects_source_document_with_versions_before_queueing(
self,
enqueue_mock,
) -> None:
version = Document.objects.create(
@mock.patch("documents.views.bulk_edit.merge_as_versions")
def test_rejects_source_document_with_versions(self, merge_mock) -> None:
Document.objects.create(
checksum="C",
title="C",
root_document=self.doc1,
version_index=1,
owner=self.user,
)
for trashed in (False, True):
with self.subTest(trashed=trashed):
if trashed:
version.delete()
response = self.client.post(
"/api/documents/merge_as_versions/",
{
"documents": [self.doc1.id, self.doc2.id],
"root_document_id": self.doc2.id,
},
format="json",
)
self.assertEqual(response.status_code, status.HTTP_400_BAD_REQUEST)
self.assertEqual(
response.data["non_field_errors"][0],
"Documents with existing versions cannot be merged into another document.",
)
enqueue_mock.assert_not_called()
response = self.client.post(
"/api/documents/merge_as_versions/",
{
"documents": [self.doc1.id, self.doc2.id],
"root_document_id": self.doc2.id,
},
format="json",
)
self.assertEqual(response.status_code, status.HTTP_400_BAD_REQUEST)
merge_mock.assert_not_called()
self.doc1.refresh_from_db()
self.assertIsNone(self.doc1.root_document_id)
-6
View File
@@ -14,7 +14,6 @@ from documents.parsers import encode_thumbnail_webp
from documents.parsers import get_default_file_extension
from documents.parsers import get_default_thumbnail
from documents.parsers import get_supported_file_extensions
from documents.parsers import is_file_ext_supported
from documents.parsers import make_thumbnail_from_pdf
from documents.parsers import rasterize_pdf_page_to_png
from paperless.parsers.registry import get_parser_registry
@@ -132,11 +131,6 @@ class TestParserAvailability:
# Test invalid mimetype returns no extension
assert get_default_file_extension("aasdasd/dgfgf") == ""
def test_file_extension_support(self) -> None:
assert is_file_ext_supported(".pdf")
assert not is_file_ext_supported(".hsdfh")
assert not is_file_ext_supported("")
class TestComputeThumbnailDpi:
@pytest.mark.parametrize(
-3
View File
@@ -39,7 +39,6 @@ class TestSanityCheckMessages:
assert messages.document_warning_count == 1
assert messages.document_info_count == 1
assert messages.global_warning_count == 1
assert messages.total_issue_count == 5
@pytest.mark.django_db
@@ -51,7 +50,6 @@ class TestCheckSanityNoDocuments:
messages = check_sanity()
assert not messages.has_error
assert not messages.has_warning
assert messages.total_issue_count == 0
@pytest.mark.usefixtures("paperless_dirs")
def test_no_issues_logs_clean(self, caplog: pytest.LogCaptureFixture) -> None:
@@ -67,7 +65,6 @@ class TestCheckSanityHealthyDocument:
messages = check_sanity()
assert not messages.has_error
assert not messages.has_warning
assert messages.total_issue_count == 0
@pytest.mark.django_db
+2 -2
View File
@@ -362,7 +362,7 @@ class TestTagHierarchy(DirectoriesMixin, APITestCase):
x = Tag.objects.create(name="X2")
y = Tag.objects.create(name="Y2", tn_parent=x)
assert y.parent_pk == x.pk
assert y.tn_parent_id == x.pk
# Moving X under D would make deepest node Y exceed depth 5 -> reject
resp_fail = self.client.patch(
@@ -383,7 +383,7 @@ class TestTagHierarchy(DirectoriesMixin, APITestCase):
)
assert resp_ok.status_code in (200, 202)
x.refresh_from_db()
assert x.parent_pk == c.id
assert x.tn_parent_id == c.id
def test_is_root_filter_returns_only_root_tags(self) -> None:
other_root = Tag.objects.create(name="Other parent")
Loaded 100 of 166 files, more files were not shown because too many files have changed in this diff. Show more