Wuff

Wikipedia wants to attract and retain editors. “Ideas are cheap, implementation costs,” but here are my thoughts anyway as a long-time but sporadic contributor. For all I know these have been discussed at length and rejected or deferred…

New pages

The original WikiWiki vision was to encourage wiki links to nonexistent pages, and if you then follow the link you jump straight into editing the missing page. Wikipedia has interposed a “Wikipedia does not have an article with this exact name” page in this workflow, and disabled the creation step for users who aren’t logged in altogether, but it’s still emphasizing page creation.

With a mature web site, most of the time people shouldn’t create new pages. The information is there, but under a different name or more likely as a section on another page, or it doesn’t qualify for inclusion. My impression is most new pages get an immediate delete or flag as Article for Deletion, and that seems extremely unmotivating (judging from Slashdotters whining about their deleted pages).

Don’t delete new pages, redirect

=> Rather than deleting, encourage redirects to sections, make it easier.

(Does Wikipedia have a policy that every significant topic in the world should have a web page? Perhaps it should, especially for languages with less coverage than English Wikipedia. It seems forward-thinking, when projects like Wikidata will allow making statements at a finer granularity than existing wikipedia pages.)

=> Instead of deleting pages, maybe add to them a redirect to a more general article (the obscure character redirects to the TV series’ page, the minor local band redirects to the musical genre, the lesser novel redirects to the novelist). The redirect page could even keep the original text but show it in a pink section “This is {{reason for deletion}} so is not its own article, hence it redirects to [[Real page]], but here is the original material“. People would be free to link to this &redirect=no page, it would remain part of the Wikipedia site, but it would not be part of the Wikipedia corpus of knowledge presented to the world. This “thanks but redirect” makes deletion less a slap in the face than an “OK, but…”. It also solves the difficulty of  seeing a deleted page’s contents. It might be a good way to let pages develop.

Editing existing pages

References preview

I usually edit a section to avoid accidental changes to other parts and because it’s faster. But in section editing there’s no references/reflist, so you have no idea if your {{cite web}} works or not.

=> when previewing a section, add an opportunistic uneditable References preview (if any) {{reflist}} that shows references. (Trick: what about references that are only spelled out in other sections — <ref name="Other Section">?)

Accidentally edits to parts elsewhere

=> It would be good if preview summarized how many “lines” you changed/added, or even tried to tell what section(s) you changed. This would encourage better edit summaries. (Note most Wikipedia markup is paragraph-oriented not line-oriented, so a straight diff is often misleading; maybe there’s a sentence-oriented diff.)

cite-o-matic sentence editor

Often when I’m editing a page I’m just updating with some new information. Imagine an editor that only preps a sentence for inclusion. It starts with

As of June 2012, your text here {{citation needed}}

If you click on the {{citation needed}} it brings up an {{cite web}} template editor that helps you build the citation, and also encourages best practices for finding original sources, permanent link, removing extraneous query string parameters, etc.

The downside is this encourages fragmentary additions rather than article-wide holistic re-editing. But does Wikimedia Foundation want more editors or better page improvements?

Social page improving

Wholescale page improvements are daunting. They’re hard to get right and the instigator ought to be soliciting feedback and participation before submitting a major page reorganization.

The coding community has solved this. On github, you can suggest an improvement as an Issue, people can chime in, and you or others can develop it in your fork of the project, and invite review, and eventually someone can pull some or all of the discrete changes comprising the update into the main version. You can also run an Etherpad or Google Docs and have real-time archived comment and chat as you improve. It’s a collaborative community process. On MediaWiki, not so much:

• you can make a version of the page as a subpage of your User:page/ , but it has no connection with the original
• you can’t diff it with the original
• nobody knows you’re working on it unless you mention it on the Talk page
• there’s no “Talk page for proposed revision Xxx”
• there’s no Page Review tool
• there’s no way to apply a series of edits from your test version of a page back to the main page
• there’s no Issue/Feature tracking for a page. (Do the current Article Feedback Tool ratings appear in a page quality section?)

Sprint Picture Mail was an easy way to upload pictures on a feature phone to a web site where you could organize them into albums. But Sprint was never a serious player in the “online photo site” business. I criticized its poor UI back in 2006, then they made some improvements that only worked in Internet Explorer, and more recently they made it into a single-URL gallery-viewer style application. Then Sprint decided to pack it in.

Amazingly, Sprint never texted my phone to say “We’re discontinuing Sprint Picture Mail”! I happened to glance at their irritating SprintZone message center on my smartphone and noticed a message about my e-mail address @pm.sprint.com going away (one I’ve never used), which led me to the announcement that the actual site is discontinuing as of April 30, 2012… which I read in May! Fortunately they have extended “Last Chance” site access for a few more weeks until June 18, 2012.

Sadly Sprint makes it hard to just grab all your pictures and videos, and you’ll lose some information without some extra work. I couldn’t find any information or a script to do the right thing, so here are my ongoing notes on the process. Please comment.

Downloading the images

Sprint has a FAQ/guide with information about downloading, but it has some omissions.

• The front page and usual log-in URL to the site just say it’s been discontinued. You must use the special log-in URL http://pictures.sprintpcs.com/downloadmymedia
• The easiest way to grab your pictures is to drag the thumbnail of an album from “My Picture Mail Albums” to the “Download to my PC” box at the top right, but this only works if the album has 50 or fewer photos! Also, this may not work for your Upload folder.
  

If an album has more than 50 pictures, it’s probably easiest to click the album thumbnail, then:

1. Ctrl+Click each thumbnail on the page until all 21 are selected
2. Drag the group to the “Download to my PC” box.
3. Name the zip file Album_name_page1.zip.
4. Advance to the next page and repeat, naming the download Album_name_page2.zip

After I’ve downloaded all the pictures in an album, I rename it on SPM to Album name (done).

File names

The individual pictures in the downloaded ZIP file get a random NNNNNNN.jpeg (or NNNNNNN.mov for videos) file name. It corresponds to the image’ identifier on SPM; while logged-in, you can view this picture at http://pictures.sprintpcs.com/i/NNNNNNN (the extension you use doesn’t matter). Normally when you view pictures on the site the image URL has stuff on the end of this specifying a size and quality and other things, e.g. /i/NNNNNNNN_75_0.jpg?outquality=56&ext=.jpg&255,255,255,1,0,0,0,0&squareoutput=245,245,245&aspectcrop=0.5,0.5,1.0,1.0,1.0

The image you see at the bare img URL is almost the same as the image file in the ZIP download, but note that it isn’t. Using the fabulous ImageMagick identify command-line tool with its -verbose option, I found an image saved this way had a quality of 90, while the downloaded ZIP had a quality of 93. So, don’t use a browser extension like “DownThemAll” to simply grab pictures from the site, you have to drag thumbnails to Download to my PC.

Unfortunately Sprint’s file name is not the file name from the camera, e.g. one featurephone I owned would name a picture taken on June 29 “2010 100629_152333.JPG”. This makes life tough if you grabbed some pictures off your featurephone by plugging it in over USB or using Bluetooth file transfer; you’ll already have some of the pictures from SPM, but under different names.

Lost metadata

JPEG files on your camera can contain all kinds of metadata: the date you took the picture, whether it’s rotated, the camera settings, sometimes the location you were in. (When you upload pictures to Facebook you should remove this identifying material, I use ImageMagick’s convert -strip option or Photoshop Element’s Save for Web….) Unfortunately I don’t have pictures from these cameras to learn which details, if any, Sprint PM tosses.

The picture files in the ZIP all have a modified date of today’s date, which sucks. They ought to have the date the files were uploaded, if not the date of the picture. The good news is the JPEG image data seems to preserve the metadata:

    exif:DateTimeDigitized: 2010:06:29 22:23:48
    exif:DateTimeOriginal: 2010:06:29 22:23:48

but the bad news is, a lot of my camera snaps don’t have this information.

Fork over the date and caption!

But SPM knows the date of a picture. If I go into the picture gallery, it shows a date for each picture, which I think is the date the picture was uploaded to SPM.

The other killer problem is, you lose any caption you put on Sprint’s site! Some of my uploaded pictures automatically got a caption of the camera’s filename, and others I had patiently named. Sprint doesn’t add this metadata to the jpeg.

So, a better workflow would be

1. download groups of pictures
2. open the album
3. advance through each picture
4. note the caption of the picture and the date taken.
5. Rename the corresponding downloaded file to match the caption, and change its modification time to reflect the date taken. And/or store this metadata in your picture gallery of choice (e.g. Windows Explorer’s File Properties).

This sort of “screen scraping” is ripe for automation. But here’s where you run into a problem with fancy web applications: as you click on albums and advance through their pictures, the URL at pictures.sprintpcs.com doesn’t change. You can’t feed a set of URLs into a command-line tool like curl or wget and then look for the caption and date, because it’s all done through onclick JavaScript wizardry at a single URL.

There must be a solution to this; whatever your browser does to get info from Sprint’s site, you can always use a computer to fake the same request. (Remember the scene in The Social Network where Mark Zuckerberg/Jesse Eisenberg codes up a Perl script to pull all the girls’ pictures from Harvard’s network?) So I used the excellent Firebug add-on for the Firefox & SeaMonkey browsers to watch my browser’s requests going to Sprint in its “Net” pane. Each time you click to advance the slide carousel, it makes a request similar to

http://pictures.sprintpcs.com/ajax/sessionExpiryCheck.do?target=/ui-refresh/getMediaContainerJSON.do%3FcomponentType%3DmediaDetail&sortCode=17&count=10&containerID=MMMMMMMM&elementID=NNNNNNNN&rnd=xxx.xxxxxx&offset=20&count=10

Sure enough, if you copy this URL into another tab,  you get a nice data structure giving details of your pictures, and crucially, the captions for them. Here’s the info for one gallery item:

{
 "elementID":NNNNNNNN,
 "containerID":"MMMMMMMM",
 "hasTransform":false,
 "creationDate":"Nov 9, 2005",
 "albumName":"Friends",
 "mimeType":"image\/jpeg",
 "audioContainerID":null,
 "hasVoiceCaption":"false",
 "audioElmtID":"",
 "description":"John Doe at reception",
 "URL":
  {"audio":"",
   "thumb":"\/i\/NNNNNNNN_40_1.jpg?outquality=56&ext=.jpg&255,255,255,1,0,0,0,0&squareoutput=255,255,255&aspectcrop=0.5,0.5,1.0,1.0,1.0",
   "image":"\/i\/NNNNNNNN_0_1.jpg?outquality=56&ext=.jpg",
   "video":"\/m\/ZZZZZZZZ_0.mp4v-es?iconifyVideo=false&outquality=56&contenttype=application\/x-quicktimeplayer"
  },
 "mediaItemNum":25,
 "mediaType":"IMAGE"
}

But there’s no need to be restricted to the URLs the web page emits. You can mess with the query string of this URL. Remove the expiryCheck wrapper, simplify the parameters (I use the URL Decoder/Encoder utility web page to convert %3D stuff into readable characters), change the offset to offset=0, and remove both count= values, and you’ll get all the info for every picture in the album. (Well, after several thousand empty whitespace characters, because the SPM programmers must be using some combo of templating and programming, and weren’t careful to disable output.)

This structure is in JSON format, and there are online viewers that present it in a more user-friendly format, e.g. http://jsonviewer.stack.hu/

I don’t know about the audio annotation and the video link. And I’m not sure whether any image effects or cropping you do on Sprint’s site affect the downloaded image, presumably that’s the hasTransform field.

Getting the albums

I have three pages of albums. The key seems to be the containerID. Maybe there’s a way to get that. Again, using Firebug’s Net pane, I see

1. http://pictures.sprintpcs.com/ajax/view/messages/getUserAlbumList.do?tabName=userAlbums&random=0.5704244691070494&initialAlbumId=TBD&mediaCount=TBD
   but that just creates an empty UI for albums.
2. Next an
   http://pictures.sprintpcs.com/ajax/sessionExpiryCheck.do?target=/ajax/view/getMediaItemsForCarousel.do…
   that seems to get a similar JSON structure for the items in my default album, “Uploads”
3. Then a
   http://pictures.sprintpcs.com/ajax/sessionExpiryCheck.do?target=/ajax/view/getAlbumListResults.do%3FsortCode%3D5%26numToDisplay%3D12%26containerID%3D%26thumbsoffset%3D0%26offset%3D0%26count%3D11&rnd=xxx.xxxxxx
   That’s the call, it returns album information.

You can get rid of the sessionExpiryCheck that’s wrapping the actual call. And as with the URL for an album’s details, take out the count, in fact take out everything, and the following simple request

http://pictures.sprintpcs.com/ajax/view/getAlbumListResults.do?sortCode=5

returns a structure listing all my photo albums, e.g.

...{
 "panel":"albums",
 "url": "javascript:YAHOO.com.sprint.pm.carousel.carouselFunctions.getMediaThumbnails('MMMMMMMM','24');",
 "imgSrc": "/i/NNNNNNNN_60_1.jpg?outquality=56&ext=.jpg&aspectcrop=0.5,0.5,1.0,1.0,1.0&showCover=main",
 "containerID": "MMMMMMMM",
 "shared": "false",
 "mediaCount":"24",
 "typeCode":"ALBUM",
 "title":"House Ideas",
 "trunTitle":"House I...",
 "coverTitle":"24 Picture(s) Updated Oct 5, 2005",
 "creationDate":"Oct 5, 2005",
 "modificationDate":"Oct 5, 2005"
}...

Alright!

For each containerID in the list,

• request
  http://pictures.sprintpcs.com/ui-refresh/getMediaContainerJSON.do?componentType=mediaDetail&sortCode=17&containerID=MMMMMMMM
• if the album has less than 50 pictures, drag it to the [Download to my PC] box on SPM,
• otherwise … maybe we can automate the download of pictures from the carousel

Triggering the download (incomplete)

Again using Firebug’s Net panel to watch the requests, when I drag a small album to the “Download to my PC” box, it requests

http://pictures.sprintpcs.com/ajax/sessionExpiryCheck.do?target=/ui-refresh/download/albumMedia.do%3FcontainerID%3DMMMMMMMM

This triggers a confirmation box telling you what to do with the download. The confirmation box also has a hidden form filled with the names and extensions of the pictures in the album, so that when you click [OK] it POSTs a request to

http://pictures.sprintpcs.com/downloadzip/sprintpictures.zip?machineid=wmtp014&

containing this information

NNNNNNNN ext=.jpeg
elementID=NNNNNNNN
…
fileName=sprintpictures
type=zip

and Sprint’s response to this is to supply the ZIP file.

The second time you drag, the download seems to trigger immediately. The URL requested is

http://pictures.sprintpcs.com/ajax/sessionExpiryCheck.do?target=/ui-refresh/download/albumMedia.do%3FcontainerID%3DMMMMMMMM

If you issue this URL request on its own, it appears to do nothing. The server responds with a web page containing just a form with the names and extensions for the pictures in the album, but the form lacks a submit button. SPM obviously expects to get this form from the server and then complete it. Automating or simulating a form submission is doable, but it’s harder than issuing a simple get request.

Other information lost

SPM can generate thumbnail for every video, something that seems beyond Windows and Linux desktops. This thumbnail isn’t included in the ZIP file when you [Download to my PC]. It is mentioned in the gallery item info in the album details, as both

"thumb": "/m/NNNNNNNN_40.mp4v-es?iconifyVideo=true&outquality=56&255,255,255,1,0,0,0,0&squareoutput=255,255,255&aspectcrop=0.5,0.5,1.0,1.0,1.0",
"image": "/m/NNNNNNNN_0.mp4v-es?iconifyVideo=true&outquality=56"

The first is a 40 pixel square thumbnail, the latter is larger. Leaving off the quality= increases image size.

It might be nice to get this video still, but the VLC media player can take a snapshot.

As you can see from my excerpted album details , SPM stores a voiceCaption and audio information. This probably doesn’t come with the download, so I should check to see if any of my pictures have these.

Understood, and slightly automated

I’ve written a bookmarklet that transforms the info about all your albums at http://pictures.sprintpcs.com/ajax/view/getAlbumListResults.do?sortCode=5 into a human-readable list of albums with links to download their metadata. If you know what a bookmarklet is, see my notes on this approach.

This could be completely automated with a script that logs in to SPM, remembers the cookie SPM sends back that authorizes your session, then requests the list of albums, then steps through requesting metadata and triggering downloads, and then even changes the date and titles of your pictures. Hmmm, there’s a tutorial for doing this sort of thing using the excellent node.js server. Again, see my fumbling notes and steps at this approach. Yet another approach is to write a web page that makes XMLHTTPRequests.