It’s been 3 years since my previous blog post about plugins. Back then, I thought it would be “soon”. I was wrong. Now I’m thinking its getting closer. This blog post therefore, communicates some of my impressions gleaned from googling the options for best practices regarding plugins.
The web is where to get unlimited new interesting things for the Moose to say. Web browsers excel at showing interesting things for viewing, and the moose gets interesting things for speaking. So theoretically similar. Web browsers have a fairly mature architecture for adding plugins, so I might learn something by looking at what they’ve done. A lot of plugin architecture is re-inventing the wheel over and over. (I’ve been computing for 30 years, I remember stuff).
– Zip files are used to bundle together all the files needed for a plugin, by Mozilla firefox, and by Google Chrome. That makes sense. Apparently though, .NET doesn’t offer zip functionality, so I may need a library like DotNetZip.
– Mozilla’s plugins rely on Javascript. Chrome’s plugins rely on HTML. (HTML that can invoke javascript) Thus, these plugins require either Firefox or Chrome to be running. My Moose can’t require that, so I have to deviate somewhat from Mozilla or Chrome standards.
– A Manifest file describes all the sub-files of a plugin and the author and version info. The Mozilla format has more stuff, the Chrome manifest has less stuff, and I think I should adopt the Mozilla, with some modifications and slight improvements. The Mozilla plugins seem more mature.
– Mozilla promoted XUL, which supposedly had better horizontal AND vertical placement of its UI buttons, text, etc. But I think HTML is good enough, and HTML5 is better. My impression of XUL is that by trying to be cross-platform, it ended up looking old-fashioned and least-common-denominator.
Update. This XUL tutorial page seems useful to me, because it nicely summarizes some of the things that my Moose plugins will need. Such as: Creating windows, adding buttons, text, images, and other UI stuff. ( but its box model, grids, stacks, etc are too complicated for my needs I think). Menus and toolbars. Handling key and mouse events, (then stuff I don’t care about: trees, DOM).
– I had high hopes 2 years ago, of using .NETs improved UI goodies of “WPF”. Maybe I should forget about XUL and HTML+CSS, and figure out if Windows 8 supports the XAML files of WPF. Oh shit:I just learned So many complexities when changing into Windows 8 apps. Darn, because I liked WPF and its use of Easing functions to make buttons come alive.
– Well, I’ve stumbled into something I didn’t know till just now. There’s a CSS 3, that is part of HTML 5, that has some “transitions” and cubic-bezier “easing functions”. So that’s good news. Now I need to find a library I can link into my project, that can make that CSS 3 come to life in my .NET windows C# code.
Update. I need a library for C# that reads CSS3 and makes WPF do it. mono.cairo might, using SVG. See perhaps the .NET WebBrowser Control, but a better option looks like Awesomium.
– Internet Explorer 10 is apparently having NO addins, so I can’t learn from them at all, UNLESS, I can put HTML5 onto all my buttons and run them simultaneously from easy .NET code.
20 best beers in the world. I don’t want to lose this link.
Oct 9, 2013 update. Photoshop has had plugins for many years. There’s probably something learnable from their API. Unfortunately I don’t have their SDK, just some googling results.
– There were 9 types of photoshop plugins. Not all are applicable to the Moose, but “Import” and “Export” would be useful (adding jokes?). The “Filter” type would be like the Moose receiving some text, and then modifying it (like removing profanity or adding personalizations). The “action” type is apparently very complex and highly capable of just about anything.
– there is a main() entry point for initialization, that receives a “selector value” parameter to handle different tasks. main() returns an error code or ‘noErr’.
– 5 types of ‘selector values’ for filter plugins.
a) ‘About’ should show an “about this plugin” dialog. (but more likely now to go to a URL)
b) ‘Parameters’ shows UI with buttons, sliders, that the user can change.
c) ‘Prepare’ grabs memory or files for initializations before a big filter calculation.
d) ‘Start’ starts the big filter calculations, validates parameters first.
(sometimes users can repeat the last filter, Command-F, without showing the Parameters dialog again.)
e) ‘Continue’ does the ‘main processing loop’
f) ‘Finish’ cleans up.
– Error reports return a string that’s added onto this phrase “Cannot complete operation because
– User Abort is allowed during filter operations, and returns a specific code -128.
– An “Advance the state of the operation” callback, allows plugins to tell Photoshop, to do more, like send another chunk of data. If plugins use this callback, they don’t need to use ‘Continue’. Instead, they do all processing in ‘Start’.
Chrome and Moxilla plugins, I think they force entry points to have a unique Namespace name as a prefix, so they don’t all have ‘main()’ as an entry point. That’s probably smart.
WordPress – of course – has plugins. Filters and Actions. Not a lot of features to boast of. Rather simple.
MySQL has plugins of 4 types: Storage Engine, Full-Text parser, Daemon, and Information Schema. It seems that full-text parser, is a filter.
Both WordPress and MySQL filter functions have the concept of priority, or, a number that determines the order that filters should run in, 1st, 2nd, 3rd, 4th, etc. I’m going to need priority/ordering of filters for the Moose.
One thing I just remembered, that I want to jot down. I want Moose’s plugins to be able to run mixtures of javascript, C# and C++. Why? Well, I need the binary code to be able to use the MMF to send messages, and I want ability to do low level memory reading from anywhere, so the Moose can gain abilities to watch what the user is doing, and joke about it.
more notes:
– NPAPI plugins (the Netscape Plugin Application Programming Interface) is being fazed out of Chrome. The most popular NPAPI plugins are: Silverlight, Unity, Google Earth, Google Talk and Facebook Video. Google’s NaCl is an alternative.
On Slashdot.org, a discussion of NaCal included this observation: “Unfortunately, the major purpose of plugins is to break out of the sandbox, to access things like sockets, webcams, local files, hardward information, etc.” I AGREE. NaCl (“Native Client”) API is very minimal. Nothing learned from that.
(an hour later…) reading about juce.com. Its manifest file has: id, name, version, description, website, license, dependencies, include, compile, and browse. The ‘browse’ feature deals with messages, timers, broadcasters, interprocess and native. ( However, Juce isn’t what my project needs)
ok, Gecko Plugin API… made my eyes bleed.
(a few hours later…) Returning to the Mozilla app manifest, and why I like it.
It has these elements: name, description, launch path, icons, developer name and url, locale info.
and optional: activities, appcache path, (these kindof suck from lameness)
and: chrome (which is specifying optional icons added for navigation)
and default_locale: “en”
and I like it because you can list different locales, (languages) and show different names and descriptions.
I think it’s lacking some CSS and font sizing that I’d like apply for different languages, when showing the plugin name and its description.
Oct 10, 2013 update. Awesomium is looking exciting. I downloaded Cheetah browser built with Awesomium, and it runs this HTML5 features website well, and on my winXP, the WpfWebBrowser.exe doesn’t work. Awesomium wins.
Today I learned about “less”, about “bootstrap from twitter” for CSS, I read more about what people are nowadays calling “responsive web design”. Good! CSS tricks will let plugins have UI that changes size to fit various screens.
Types of Moose Plugins:
Here is my attempt to group my plugins into categories, to plan what I need their API to support:
a) Fetch from internet, something interesting: RSS newsfeeds. Weather reports, Web page scrape, satellite spotting, Shop bargains on Craigslist, eBay or Kijiji. Speak facebook, Speak twitter tweets,
b) Things to Say: (built-in, can run without internet access): Hello at Startup, Goodbye at Shutdown, Say the time, on the hour, Say name of windows, Jokes, Diet reminder. Sunrise/sunset times, moon phases, say happy birthday, report traffic at shutdown, Calendar and Time reminders. Speak clipboard text. Speak webpage text to the blind, Say special days (like Happy secretaries day). Remind to bathe/clip fingernails, File-system-watcher for new files or changing files, Say something from text input and record video of it.
c) Affect the Moose: Change position of moose. Change Voice and speed. Shh options, Boss-Key, disable internet, disable moose during certain games or watching video. Change entrance and exit effects, Change Moose size. Manage updates. Zoom in on moose, Hook up Kinect for face tracking. Choose a Chatbot, Choose a Siri-like information source, Choose an Agent for automating windows actions, Manage attachments (hats, ties, necklaces), Setup online payments, Shop for Moose add-ons, Customize a gift moose for a friend,
d) Info about user: Name, address, gender, birthday, timezone, GPS location, email, locale and language, Setting users email servers, Oath credentials. Billing info for paying for our monitization features.
e) Filters: Add personal name into phrases (eg “Hey Steve, its 3 oclock pm”), filter profanity (remove “fuck”), add profanity (sometimes its funny to insert swearing), substitute phonetical spellings for troublesome words. (NOTE: Filters need the ability to run in a certain order)
f) Show Moose Status: List of what he just said, Send jokes by email, or post to twitter or post to facebook or weibo. Get HELP or read Tips, Look at rewards earned. Show a forum for chatting with other users, Show leaderboards, Submit new jokes, Vote on jokes others submitted,
Analysis of my plugin needs:
Types a) and b) need selector values for pose the question, “Is it time to speak this yet?”
i) If Moose.exe handles time-keeping, then these plugins would only get called when it IS time to speak. So they’d need to tell Moose.exe how much time to elapse. like 1 minute, 30 minutes, 1 hour, etc. OR they’d tell Moose.exe the exact time they want to be called next.
ii) If the plugin handles time-keeping, then lots of CPU cycles are wasted as each plugin checks the time every 32 milliseconds. I want to avoid that.
iii) If plugins let windows set a timer, that triggers a signal to call the plugin, that’s OK too.
Types a) and maybe b), pose themselves the question, “Has anything changed on the web? Is there anything new to say?” The internet plugins certainly must save what they fetched, and save what they spoke, so they can do their own comparisons to see if anything has changed.
All plugin types have settings, some stored locally, some stored on our server. Maybe both. Storage and retreival of settings is a common need.
Some plugins might have real-time animation needs, other plugins wouldn’t need any CPU time for static simple HTML and the mouse not hovering overtop. Maybe our plugins need a property that tells Moose.exe how much attention it needs, even if its for a different purpose from deciding whether to speak.
All plugins may need to show something in a visual notification window above the sys tray. They need to be able to do that, without making the whole control panel come to the foreground. We ought be able to let them show anything HTML5 can do.
October 11, 2013 update. OK. I need to think about this. My own controls, which have quite a long list of abilities, will be providing quite a few of these plugins that ship with the Moose. For monitization purposes, most of the plugins will have some features usable, and some that are disabled in the free default version. I need a way for users to pay a dollar, to enable certain features. I need these HTML5 pages to be able to do that secure communication that authorizes paid-for features to become activated.
I want the Moose to disappear when not speaking. But when he/she pops into view and is speaking, is an opportunity, to show something in the notification area above the sys tray. Needing to monitize, needing to encourage social sharing of jokes, I need to plan “variety” into what is shown there. Question: DO I need plugins to also contribute to this? Can 3rd party plugins entice users to click to go to their websites?
Initializing a plugin, is actually 2 different things. 1) is initializing the part that constantly runs without any UI showing, to decide what and when to send a phrase to speak. 2) is initializing the parts that will show UI controls in the control panel.
Oct 12, update I have a few minutes to brainstorm about future coding.
1) When my application Moose.exe is starting up, its going to look all the available plugins in a certain folder, and it needs to initialize each one.
– The first thing it should do is read an XML file for an ON/OFF setting. This is just in case the plugin’s own javascript or C# code is become incompatible after a moose upgrade. We need a way to disable a plugin before a plugin gets a chance to run its own code.
– If the plugin is ON, we call its initialize code for waiting-watching-speaking. We’ll allow Javascript or compiled C# or C++. There might be both javascript and compiled binary, so I need a setting to indicate which gets called first, which gets called second.
– The waiting-watching-speaking will initialize itself, read its own saved settings, load its strings and return OK or error codes. It needs to somehow return to Moose.exe, what it will need for frequency of receiving CPU time,
2) When the main-event-loop of Moose.exe running, we need to give time to the plugins periodically. This is for plugins that might speak something. (filter plugins, which don’t generate speakable content on their own, don’t use this). Again, we might call a Javascript entry point, or a C# or C++ compiled entry point, and we need a setting to say which comes first. For simple things, it will just be javascript.
3) When a Plugin creates something to be spoken, it will return a string to be spoken, and our Say() function will place it on a queue, waiting for a chance for the 3D moose to speak it. In addition to just a string to be spoken, we need to potentially show monitization graphics and buttons in the notification area above the sys tray. I think we should show mini HTML5 windows, which can have all sorts of graphics and active buttons. So, we need to receive a HTML5 document somehow. Question: Do we get a URI to a file, or receive a big string of HTML5? What other parameters do we need to position the popup window and give it a nice small size that adapts to fit the content?
tricky stuff: Normally when the moose pops onto the screen, we don’t want keypresses to go to moose windows. We want the user to keep typing where-ever they were. Otherwise it would be a disaster. So an HTML5 notificiaton window above the systray, can’t steal keypresses. Also, it should fade in, persist while the moose is talking, and then fade away (unless the user is hovering the mouse over a button).
4) The Moose’s Control Panel doesn’t show itself until a user chooses it from the Sys Tray. When they Do, the control panel needs to call each plugin, to receive some information to display their Name, (Font,size,direction), Description (font, size, direction), Icon. Maybe these, plus an optional HTML5 mini page that only shows stuff on a button. I think one or the other. If HTML5, then it must look after displaying everything. ( I need to figure out, if Awesomium can tolerate to display 20 different HTML5 mini pages. ) It would be good, for my control panel buttons to show
hover effects and mousedown effects.
5) Filter plugins, will receive a parameter containing text to be spoken, and they will return modified text to be spoken, or perhaps, no modification. Simple filters can be simple javascript. Complex ones could refer to a web server, for instance, to determine if there is some hacked spam trying to be spoken. Some filters might want to look up hashtags, and replace them with better speakable words. Some filters might follow embedded http links, and use some logic to speak a page title, or a picture alt, or who knows.
6) Each plugin will have UI, specified by an html5 page hopefully. The name of buttons, radiobuttons, slider strengths, etc, will be read from a plugin settings file. Later, settings are saved to the file. Is the file local, or on our webserver, or on a plugin-3rd-party-webserver?
Oct 17, 2013 update. I’m making notes, starting to search for a web host outside the influence of US monitoring. When I have free time, I’ll return to investigate Namecheap.com, Cloudflare.com, akamai.com, edgecast.com, artmotion, koddos.com. I read that Norway, Finland, Netherlands and possibly Swiss, are good locations to consider.
Oct 24, 2013 update. Today I’m looking at the Flash .swf format, to see what they’ve learned.
– The header starts with an 8 byte “signature” where ‘FWS’ is an uncompressed file, ‘CWS’ uses ZLIB compression, and ‘ZWS’ uses LZMA compression. This is possibly a good idea, to accommodate future compression standards too.
– The data is subdivided into blocks (called Tags), each one independent of the others, and data elements in one block aren’t allowed to reference other blocks.
– Flash includes a ‘Drop Shadow filter’, which I should investigate.
– The Font tags store lots of parameters about a font: its ID, name length, name, ShiftJIS character codes, ANSI character codes, Italics, bold, and Glyph to code tables. Language ID, Ascender height, descender height, leading height, kerning table stuff. WOW. ( I may need this, because my plugins need to show a name and description,)
– Some Button events: IdleToOverUp, OverUpToIdle, OverTopToOverDown, OverDownToOverUp, … OUtDownToOverDown, OverDownToOutDown,OutDownToIdle,
– Button blend mode, and a lot more about buttons.
( I won’t let individual plugins have non-standard behaviors, but this reminds me to give all of them, LOTS of “character”).
Oct 26, 2013 update. Today I realized there’s possibly another kind of plugin that I need. “What is the user doing now?” There are lots of possible ways to figure out what the user is doing. In web browsing, we could create a complementary plugin for each web browser, that communicates to the moose plugin. Or, it could be like a screen OCR reader, that periodically finds words on the whole screen, and generates a few keywords from what it recognizes. Or it could be a plugin that checks what is the foreground app, and compares it to a web-server that keeps a database of applications, and tells us if its a game, a music player, watching a video, etc. I don’t want a multitude of speakable phrases coming in from web browsers. I’d prefer to just receive keywords conveying concepts, and then let the Moose create jokes, or chatbot conversation about it. Anyway, this type of plugin is free to implement inter-application communication with other web-browser plugins, and we need to create a standard place for these to communicate to the Moose, what’s going on.
