A Digital Workflow for Academic ResearchDevonThink Archives » A Digital Workflow for Academic Research

Create universal indexes for your own books and papers

Almost every academic book has at the end of it an index ,which helps the reader find in it pertinent material regarding the indexed words or phrases. Using a more technical terminology, I can say that an index is a list of “dead” analog internal links (If you want to make them active you can use AutoBookmark, but this is for another post).

Nowadays it is quite probable that any young academic researcher has accumulated a considerable amount of academic sources in PDF format. If he uses Mac computers, DEVONthink and Skim from now on he can create universal indexes. By universal indexes I mean that the active links are not internal to a specific source but external to any document that contains the indexed word or phrase and is in a DEVONthink database. You can see an example of a part of a universal index in MediaWiki format in the following image:

The above text is produced automatically and you can see in it some of the conventions that I use in my workflow. For example the names of my PDF files follow the pattern “title – last name” and the corresponding MediaWiki page hat the “@” in front of the PDF name. The links named “correct logic” open the corresponding source in DEVONthink and all the instances of “correct logic” are highlighted. An example of a URL of these links is the following: x-devonthink-item://A6A8BAC2-B6E7-4568-A117-C74335C4C7BC?search=correct_logic.

Some advantages of universal indexes are the following:

You can immediately see all the instances of the indexed word or phrase which are in each document.
You can copy a part or the whole of it in one or more documents.
You can freely annotate the index because it is just plain text. For example sometimes I find it quite useful to copy and paste the relevant excerpt from each page as is shown below:

The AppleScript which can create universal indexes is the following:

Click here to see the source code

--http://www.organognosi.com
-- 2012-09-30
--it matches exact phrases using quotes and single words
--the locked files in DEVONthink are excluded from the search
--the index is created for the active DEVONthink database
tell application "DEVONthink Pro"
set searchWord to text returned of (display dialog "Enter the search word/phrase:" default answer "")
set searchWordList to every word of searchWord
set numberOfSearchWords to count searchWordList
set the searchWordWithoutSpaces to replaceText(searchWord, " ", "_") of me
set searchWordQuoted to "\"" & searchWord & "\""
set resultList to search searchWordQuoted in current database locking no
set numberResults to count resultList
if numberResults = 0 then display dialog "No matches were found." buttons {"Cancel"}
set numberOfDocuments to text returned of (display dialog "The found docs are " & numberResults & ", please set the numbers of docs for processing:" default answer numberResults)
if numberResults > numberOfDocuments then set resultList to items 1 thru numberOfDocuments of resultList
set RefType to my chooseRefType({"MediaWiki", "RTF"})
if RefType is 1 then
set indexLinks to "===Universal index of \"" & searchWord & "\"===
number of sources:" & numberOfDocuments & " out of " & numberResults & " matches - " & (the current date) & "
"
else if RefType is 2 then
set indexLinks to "<html>
<body>
<h3>Universal index of \"" & searchWord & "\"</h3>
number of sources: " & numberOfDocuments & " out of " & numberResults & " matches - " & (the current date)
end if
repeat with book in resultList
set pathBook to path of book
set referenceURL to reference URL of book
set nameBook to name of book
set wordInstances to ""
tell application "Skim"
try
open pathBook --κολλάει εάν κάποια αρχεία δεν υπάρχου
tell document 1
set all_pages to count of pages -- so we only calculate it once
if RefType is 1 then
set bookTitle to "
====" & nameBook & "====
[" & referenceURL & "?search=" & searchWordWithoutSpaces & " " & searchWord & "] in [[@" & nameBook & "]]<br />
"
repeat with pageNumber from 1 to all_pages --
set DEVONthinkPageNumber to pageNumber - 1
if numberOfSearchWords ≤ 1 then
set pageText to (get text for page pageNumber) -- suck up the text into a string!
ignoring hyphens
set pageText to (every word of pageText) as list -- tokenise!
end ignoring
if pageText contains searchWord then -- record page number if word on page
set wordInstances to wordInstances & "[" & referenceURL & "?page=" & DEVONthinkPageNumber & "&?search=" & searchWordWithoutSpaces & " " & pageNumber & "] "
end if
else
set pageText to (get text for page pageNumber) as Unicode text -- suck up the text into a string!
if pageText contains searchWord then
set wordInstances to wordInstances & "[" & referenceURL & "?page=" & DEVONthinkPageNumber & "?search=" & searchWordWithoutSpaces & " " & pageNumber & "] "
end if
end if
end repeat
else if RefType is 2 then
set bookTitle to "<h4>" & nameBook & "</h4>
<a href=" & referenceURL & "?search=" & searchWordWithoutSpaces & ">" & searchWord & "</a>: "
repeat with pageNumber from 1 to all_pages --
set DEVONthinkPageNumber to pageNumber - 1
if numberOfSearchWords ≤ 1 then
set pageText to (get text for page pageNumber) -- suck up the text into a string!
ignoring hyphens
set pageText to (every word of pageText) as list -- tokenise!
end ignoring
if pageText contains searchWord then -- record page number if word on page
set wordInstances to wordInstances & "<a href=" & referenceURL & "?page=" & DEVONthinkPageNumber & ">" & pageNumber & "</a> "
end if
else
set pageText to (get text for page pageNumber) as Unicode text -- suck up the text into a string!
if pageText contains searchWord then
set wordInstances to wordInstances & "<a href=" & referenceURL & "?page=" & DEVONthinkPageNumber & ">" & pageNumber & "</a> "
end if
end if
end repeat
end if
end tell
close document 1
end try
end tell
if RefType is 1 then
set indexLinks to indexLinks & bookTitle & wordInstances & "
"
else if RefType is 2 then
set indexLinks to indexLinks & bookTitle & wordInstances & "
</body>
</html>"
else if RefType is 3 then
end if
end repeat
if RefType is 1 then
set the clipboard to indexLinks
else if RefType is 2 then
set the clipboard to indexLinks
tell application "Finder"
do shell script "pbpaste | textutil -stdin -format html -convert rtf -stdout | pbcopy -Prefer rtf"
end tell
end if
end tell
on chooseRefType(typeList)
tell application "Skim"
set theResult to choose from list typeList with prompt "Reference type:" default items {"RTF"}
if theResult is false then return 0
set refTypeNumber to theResult as string
if refTypeNumber is "MediaWiki" then
return 1
else if refTypeNumber is "RTF" then
return 2
end if
end tell
return RefType
end chooseRefType
on replaceText(thisText, searchString, replacementString)
set AppleScript's text item delimiters to the searchString
set the itemList to every text item of thisText
set AppleScript's text item delimiters to the replacementString
set thisText to the itemList as string
set AppleScript's text item delimiters to {""}
return thisText
end replaceText

The result is sent to the clipboard. The script is generally slow especially when there are a lot of matched documents. The index is created either in MediaWiki format or RTF format.

Sep 2012

AUTHOR John Sidiropoulos

CATEGORY

all my posts, AppleScript, DevonThink, Skim

COMMENTS No Comments

Create DEVONthink bookmark for the current PDF page

The AppleScript which performs this function is the following:

Click here to see the source code

--http://www.organognosi.com
tell application "DEVONthink Pro"
set nameCurrentPDF to name of content record of window 1
set targetgroup to parent of content record of window 1
set AppleScript's text item delimiters to {""}
set charFullStop to character -4 of nameCurrentPDF
if charFullStop is "." then set nameCurrentPDF to characters 1 thru -5 of nameCurrentPDF as string
display dialog "Enter bookmark name:" default answer nameCurrentPDF buttons {"Send bookmark to current group", "Select the target group", "Cancel"} default button 2
copy the result as list to {nameBookmark, selectedDestination}
set DEVONthinkPageNum to current page of window 1
set realCurrentPageNum to DEVONthinkPageNum + 1
set DEVONthinkPage to DEVONthinkPageNum as text
set realCurrentPage to realCurrentPageNum as text
set fileLink to the reference URL of content record of window 1
if selectedDestination is "Send bookmark to current group" then
create record with {type:bookmark, name:nameBookmark, URL:fileLink & "?page=" & DEVONthinkPage} in item 1 of targetgroup
else if selectedDestination is "Select the target group" then
create record with {type:bookmark, name:nameBookmark, URL:fileLink & "?page=" & DEVONthinkPage} in display group selector
else if selectedDestination is "Cancel" then
--do nothing
end if
end tell

The bookmark is created for the current PDF page in the frontmost DEVONthink window. After executing the script, the following window is shown:

You can set the bookmark name and choose the group in which the bookmark will be sent.

Special thanks to R.K. for providing ideas and support for the development of this script.

Aug 2012

AUTHOR John Sidiropoulos

CATEGORY

all my posts, AppleScript, DevonThink, PDF

COMMENTS No Comments

Integrate Microsoft Word with DEVONthink

As you may have noticed, DEVONthink links are not properly recognized by Microsoft Word.

First of all, when you have one of them (e.g. the “x-devonthink-item://14AE6E1A-DB3E-429B-8A46-4E1C3174A6F2?page=23″) in the clipboard by using the “Copy Page Link” DEVONthink command and paste it to a Word document, you do not get a proper hyperlink but you just see the raw URL. Moreover, when you try to create a Word hyperlink using the normal procedure, (Insert -> Hyperlink… or Command + K) the DEVONthink URL is changed to a degree that cannot be recognized and used by Word. Specifically, after inserting it in the “Link to:” field of the “Insert Hyperlink” Word window

it is converted to “file://localhost/x-devonthink-item/::14AE6E1A-DB3E-429B-8A46-4E1C3174A6F2%3Fpage=23″.

Fortunately the changes are not so destructive and the link can become functional using… what else, an AppleScript!

The only requirement for the proper execution of the script is to have either the cursor inside the hyperlink or the hyperlink should be selected with or without any of its adjacent text. If more than one hyperlinks are in the selected text then the first link will be activated by the AppleScript.

The AppleScript code for activating a link in Word is the following:

Click here to see the source code

tell application "Microsoft Word"
	set linkList to hyperlink objects of selection
	try
		set linkFirstItem to item 1 of linkList
	on error
		display dialog "Please select a hyperlink in Word before executing the script."
	end try
	set linkAddress to hyperlink address of linkFirstItem
	set linkAddressText to linkAddress as text
end tell
 
set pagePosition to offset of "page=" in linkAddress
set uuidLink to characters 38 thru -1 of linkAddress as text
 
if pagePosition = 0 then
	set thinkWindowName to uuidLink
	set xdevonLink to "x-devonthink-item://" & uuidLink
else
	set thinkWindowName to replaceText(uuidLink, "%3F", "?") of me
 
	set xdevonLink to "x-devonthink-item://" & replaceText(uuidLink, "%3F", "?") of me
end if
 
tell application "DEVONthink Pro"
	open tab for URL xdevonLink
	close think window named thinkWindowName
	activate
end tell
 
on replaceText(thisText, searchString, replacementString)
	set AppleScript's text item delimiters to the searchString
	set the itemList to every text item of thisText
	set AppleScript's text item delimiters to the replacementString
	set thisText to the itemList as string
	set AppleScript's text item delimiters to {""}
	return thisText
end replaceText

Click here to open the source code in AppleScript Editor

Now that we can use DEVONthink links in Word it is time to automate the creation of Word hyperlinks! The new hyperlink will have as destination the selected item in DEVONthink and it is created at the cursor position in the front Word document. The display text of the link follows the template: “(Name of DEVONthink record)” or “(Name of PDF file: Page Number)”.

The AppleScript code which creates Word links to a DEVONthink record is the following:

Click here to see the source code

tell application "DEVONthink Pro"
	set viewedRecord to the content record of viewer window 1
	set linkRecord to the uuid of viewedRecord
	set kindRecord to kind of viewedRecord
	set titleRecord to name of viewedRecord
 
	if kindRecord is "PDF" or kindRecord is "PDF+Text" then
		set PdfPage to current page of viewer window 1
		set DEVONthinkLink to "x-devonthink-item://" & linkRecord & "?page=" & PdfPage
		set the clipboard to DEVONthinkLink
 
		tell application "Microsoft Word"
			activate
 
			set textSelected to (get text object of selection)
			set startRange to start of content of textSelected
			set endRange to end of content of textSelected
			set theRange to create range active document start startRange end endRange
			tell active document to make new hyperlink object at end with properties {text to display:"(" & titleRecord & ": " & PdfPage & ")", hyperlink address:DEVONthinkLink, text object:theRange}
		end tell
 
	else
		set DEVONthinkLink to "x-devonthink-item://" & linkRecord
		set the clipboard to DEVONthinkLink
 
		tell application "Microsoft Word"
			activate
 
			set textSelected to (get text object of selection)
			set startRange to start of content of textSelected
			set endRange to end of content of textSelected
			set theRange to create range active document start startRange end endRange
			tell active document to make new hyperlink object at end with properties {text to display:"(" & titleRecord & ")", hyperlink address:DEVONthinkLink, text object:theRange}
		end tell
	end if
end tell

Click here to open the source code in AppleScript Editor

May 2012

AUTHOR John Sidiropoulos

CATEGORY

all my posts, AppleScript, DevonThink, hyperlink, Word

COMMENTS 2 Comments

Count the number of pages of selected PDF files in your DEVONthink database

I wrote the following AppleScript in order to satisfy my curiosity about this number!

Click here to see the source code

tell application "DEVONthink Pro"
set these_items to the selection
if these_items is {} then error "Please select some contents."
set totalNumber to 0
repeat with this_item in these_items
set numPages to page count of this_item as integer
set totalNumber to totalNumber + numPages as integer
end repeat
display dialog "The total number of pages of selected PDF files is " & totalNumber
end tell

Mar 2012

AUTHOR John Sidiropoulos

CATEGORY

all my posts, AppleScript, DevonThink

COMMENTS No Comments

MediaWiki, Mac and custom URL schemes

A lot of Mac applications have their own URL schemes. In the following table you can see some examples of custom URL schemes:

Application	Custom URL scheme
Evernote	evernote://
DEVONthink	x-devonthink://
BibDesk	x-bdsk://
OmniFocus	omnifocus://
VoodooPad	x-voodoopad-item://
TextMate	txmt://

If you want to see all URL scheme handlers on your system you can install the freeware app RCDefaultApp. Also handleopenurl.com maintains a URL scheme index of more than 600 iOS apps.

Custom URL schemes are extremely valuable because you can address specific data of an application by using them. They are the basic building blocks of all hyperlinks. In my post “Automated summary creation linked with the source PDF file” you can see how I use x-devonthink URL schema in order to connect my notes with the source PDF file.

Moreover, as you can see in my post “Embed AppleScripts in a MediaWiki page” there is a custom URL scheme for AppleScript! In my next posts I will publish a series of AppleScripts, which use the “applescript://” URL scheme.

MediaWiki can recognize all these custom URL schemes if you add the following lines in the LocalSettings.php file:

$wgUrlProtocols[] = “evernote://”;
$wgUrlProtocols[] = “devonthink://”;
$wgUrlProtocols[] = “x-bdsk://”;
$wgUrlProtocols[] = “omnifocus://”;
$wgUrlProtocols[] = “x-voodoopad-item://”;
$wgUrlProtocols[] = “txmt://”;
$wgUrlProtocols[] = “applescript://”;

You can see my post “How to use DevonThink links in a MediaWiki page” for more detailed information.

Dec 2011

AUTHOR John Sidiropoulos

CATEGORY

all my posts, AppleScript, BibDesk, DevonThink, Mac, MediaWiki, TextMate

COMMENTS No Comments

Send DEVONthink tags to BibDesk as keywords

Continuing in the same spirit as in the previous post, I wrote an AppleScript for sending DEVONthink tags to BibDesk. Only one item should be selected in DEVONthink and BibDesk. No duplicate keywords are created in BibDesk.The AppleScript code is the following:

Click here to see the source code

tell application "BibDesk"
--only one item should be selected in BibDesk
set thePub to the selection of document 1
set keywordsPub to keywords of (item 1 in thePub)
set old_delimiters to AppleScript's text item delimiters
--the space inside quotes is very important!
set AppleScript's text item delimiters to ", "
set keywordsPubList to text items of keywordsPub
set AppleScript's text item delimiters to old_delimiters
end tell
�
tell application "DEVONthink Pro"
set thisSelection to the selection
if thisSelection is {} then error "Please select something"
if (length of thisSelection) is greater than 1 then error "Please select only one item"
set the clipboard to ""
set thisItem to (item 1 of thisSelection)
set tagsItem to tags of thisItem
repeat with x from 1 to the count of tagsItem
set positionOfDuplicate to list_position(item x of tagsItem, keywordsPubList) of me
if positionOfDuplicate is 0 then
set keywordsPub to keywordsPub & ", " & item x of tagsItem
else
set keywordsPub to keywordsPub
end if
end repeat
end tell
tell application "BibDesk"
if the first character of keywordsPub is "," then
set keywordsPub to text 3 thru -1 of keywordsPub as text
end if
set keywords of (item 1 in the thePub) to keywordsPub
end tell
�
--subroutine from "AppleScript 1-2-3" p.567 for checking for duplicates
on list_position(this_item, this_list)
repeat with i from 1 to the count of this_list
if item i of this_list is this_item then return i
end repeat
return 0
end list_position

Jun 2011

AUTHOR John Sidiropoulos

CATEGORY

all my posts, AppleScript, BibDesk, DevonThink

COMMENTS No Comments

Convert DEVONthink tags to MediaWiki categories

DEVONthink has a quite efficient tagging capability. Especially, the autocomplete feature of the tag bar, at the lower part of a DEVONthink window, saves me a lot of time and provides increased consistency in my tags.Moreover, I maintain a separate page in my wiki for each book or paper that I have read. There I write all my notes and thoughts about each source. MediaWiki’s name for tag is category and the required markup code for it is the following:

[[Category:tagName]]

As I did not want to insert manually the tags for each source twice I wrote an AppleScript, which converts DEVONthink tags to MediaWiki categories. One item should be selected in DEVONthink before executing the script. The result is sent to the clipboard.

The AppleScript code is the following:

Click here to see the source code

--2011-06-18
    --http://organognosi.blogspot.com
 
tell application "DEVONthink Pro"
    set thisSelection to the selection
    if thisSelection is {} then error "Please select something"
    if (length of thisSelection) is greater than 1 then error "Please select only one item"
    set the clipboard to ""
    set newCategories to ""
    repeat with thisItem in thisSelection
        set tagsItem to tags of thisItem
        repeat with eachTag in tagsItem
            set tagForCategory to eachTag
            if tagForCategory is in {"Linked with MediaWiki"} then
                set newCategories to newCategories
            else
                set newCategories to "[[category:" & tagForCategory & "]] "
                set the clipboard to (the clipboard) & newCategories
            end if
        end repeat
    end repeat
end tell

Jun 2011

AUTHOR John Sidiropoulos

CATEGORY

all my posts, AppleScript, DevonThink, MediaWiki

COMMENTS No Comments

A Digital Workflow for Academic Research
Tools for Knowledge and Knowledge for Tools

Recently Twitted

All my posts

Tags

Blogroll