SPE Archives - Flux Digital Sitecore Blog

An Generative AI Module for Sitecore XP – S-3PO

About 6 weeks ago I posted on Linked in and Twitter with a GitHub link to a new Module I’d just finished creating: S-3PO – An Generative AI Module for Sitecore XP – built with SPE and using various models from ChatGPT/OpenAI.

Shortly after that I started a new role at SoftServe, so have been pretty busy settling in and not had much time to Blog. However I thought it was about time I wrote a Blog Post on what S-3PO is so others can find it and explain what it does and why I built it.

TLDR: Download S3PO – An Generative AI Module for Sitecore XP

Why S-3PO?

Well even if your not a Star Wars fan most people have heard of C-3PO right? the awkward gold droid from Star Wars that follows Luke around trying to help. Well that’s the inspiration here. I’m a big Star Wars fan so this was an obvious name for me.
AI has intrigued me for a few years now and up until recently I’d never tried to build anything much with AI. Generative AI seems to be ubiquitous now so I decided it was time to learn more about it.

What does S-3PO do?

The main focus of the module in the first version is around helping content editors with content entry. After installing the module you will see two icons under the review tab.

Note you will need to add your OpenAI API Key within the settings after installing the module. You can also configure which Text Model and Image Models to use here too along with the Media Library folder to save the images to.

It is worth experimenting with different models to see what results you get as some return better results but are also more expensive than others. The current functionality is as follows.

Rewriting Text with AI

Ever wanted to re-word what you’ve written but struggling to understand how to word it?
Well S-3PO to the rescue. It generates up to 3 versions for you; using GPT-35 Turbo, GPT-4o mini, GPT-4o Turbo or GPT-4o (depending on what your choose in the settings).

Here is a demo of the re-write functionality:

Generate Image with AI

If you find yourself needing an image for content your writing then S-3PO can do that too. It will Generate you up to 3 Images using Dall-e-2 or Dall-e-3 (depending on what your choose in the settings).

Note: Dall-e-3 is limited to an single variation.

Here is an demo of using the Image Generation functionality:

How do I install it?

Go to the download page here and download version 1 here: https://github.com/fluxdigital/S-3PO/releases/tag/1.0.0.

Follow the install instructions here: https://github.com/fluxdigital/S-3PO?tab=readme-ov-file#pre-requisites

Note: you must have Sitecore PowerShell Extensions installed.

What Next?

You may have seen the recent announcement of Sitecore Stream at Sitecore Symposium, if not have a read as there is very impressive list of features planned: AI workflows, generative co-pilots, and brand-aware AI and more. I’m excited to see more on Stream in the coming months.
I’m obviously not looking to emulate this but this is just the first version of the Module, I plan to release future versions adding more features to assist content editors in the near future.

I’ve really enjoyed the playing with AI technologies and learning about it and I hope it’s useful to others who want to add AI feature for free to Sitecore XP. Watch this space for updates coming soon and feel free to feedback on anything.

Sitecore Page Exporter

Something I need to do regularly is pull down a page from an higher environment (such as UAT or Production) to my local machine or Test. I’ve done this in the past by manually building packages, using Sitecore Sidekick or SPE’s ‘Quick Download Tree as package’ option.

However the SPE’s package option does not support packaging up the datasource items (unless they are child items of the page). In my experience there are often global datasources that are not sub-items of the page. This can take quite some time to do manually, especially for large pages.

Enter Sitecore Page Exporter

So I decided to create ‘Sitecore Page Exporter’ using SPE which will handle this. It supports exporting a specific page as an package and optionally the datasources, images and sub-items. This is v1 so I plan to add more features in the near future.

Pre-requisites

You must have Sitecore PowerShell Extensions installed. This release has been tested with Sitecore 10.3 and SPE 6.4 but should work with older versions also.

Install Notes

Download the v1 package from the release link
Install the package using the Sitecore package install option in the Sitecore Desktop
You should now have Sitecore Page Exporter installed under the SPE module:

Usage

To export an page right-click the page in the Content Editor and choose: Scripts > Export Page as Package:

The following options are then available to you:

Choose your options and Click ‘OK’
Download and save the package

You get an overview of the export if you click ‘view script results’:

You will get an summary at the end of the number of items included also:

Upload the package to where you want to use the page (e.g your development machine)

Hopefully this is useful for others too. Let me know of any features you think might be added or any issues you have with this.

Bulk Enable/Disable Sitecore Users with SPE

We’re currently pretty close to completing an upgrade to Sitecore 10.3 for a client and during the go live process we needed to disable most of the users apart from a few admin users and then re-enable them again after go-live.

We have a lot of users in the system and so I turned to Sitecore PowerShell Extensions (SPE) to automate this process. Here is the script I came up with:

When you run the script it has a dialog which allows you to select if you would like to enable or disable users and to choose which Admin users you would like Exclude when running the Disable/Enable:

Obviously you don’t want to accidently lock yourself out of Sitecore by disabling the main sitecore\Admin user!, therefore I’ve put a check in for this to try and stop this happening:

Once the script has completed you will see a modal confirming the number of users Disabled/Enabled:

Then you will be shown a report showing a list of all the users that have been either Enabled or Disabled:

Note that as I unchecked the sitecore\testadminuser in the modal dialog it has disabled this user along with all the other non-admin users in Sitecore.

These screenshots are from my local dev environment, but I’ve tested this script on hundreds of users and it runs in a few seconds.

Hopefully it’s useful for others who need to do something similar and can be easily updated too.

Deleting IAR items from the Database & Content Editor warnings for over-written IAR Files

Having recently created an Sitecore 10.3 IAR Package for the Scheduled Publishing Module I needed to remove the files from the database as they were still there even though they are now in the .dat files I created.

In previous versions of Sitecore it was quite tricky to do this but luckily were using Sitecore 10.3 and the Sitecore CLI has been updated to allow us to delete specific items from the database with the itemres cleanup command.

The commands we need to run are as follows:

dotnet sitecore itemres cleanup -p "/sitecore/templates/Scheduled Publish" -r

 dotnet sitecore itemres cleanup -p "/sitecore/system/Tasks/Schedules/ScheduledPublishTask" -r 

 dotnet sitecore itemres cleanup -p "/sitecore/system/Tasks/Commands/ScheduledPublishCommand" -r 

 dotnet sitecore itemres cleanup -p "/sitecore/system/Modules/Scheduled Publish" -r 

 dotnet sitecore itemres cleanup -p "/sitecore/content/Applications/Content Editor/Gutters/Scheduled Publish" -r 

 dotnet sitecore itemres cleanup -p "/sitecore/content/Applications/Content Editor/Ribbons/Strips/Publish/Scheduled Publish" -r 

 dotnet sitecore itemres cleanup -p "/sitecore/content/Applications/Content Editor/Ribbons/Chunks/Scheduled Publish" -r 

 dotnet sitecore itemres cleanup -p "/sitecore/system/Field types/Custom Field Types" -r

It’s possible to run these commands using the ‘what if’ flag (-w), to see what would happen if you ran them which is quite handy for testing them first. You will see a message saying that no changes will be made:

Note that unfortunately It’s not possible to run the ‘what if’ if not providing a path. It seems this might be coming in 10.4:

Once you’ve run the commands properly (without the -w switch) then you will see confirmation that the item(s) were removed like so:

The next step was that I wanted to check the above deletes have worked correctly and that all the items were indeed coming from the IAR files and not from the database.

I decided an Content Editor warning would be a good way of doing this, I have created these using SPE before so had a look around and found this really useful post from Jan Bluemink on doing this for IAR files. It mostly worked ok but the code that was share had some issues with the formatting and I wanted to make some improvements. Here is my updated version:

Note: to use this you need to ensure that your script library is configured as an ‘PowerShell Script Module’, that the integration points for Content Editor Warnings are enabled and the script placed in the correct sub-folder (Warning).

The script displays an Content Editor blue Info message if an item is coming from an IAR file and hasn’t been over-written like so:

And if it has been over-written (is coming from the database) then it shows an orange warning message like so:

This was really useful for confirming that the IAR files were working as expected. I actually set this up before running the cleanup commands above so that I could check I was getting the Orange message initially and then the Blue one after running the cleanup commands.

You can test this yourself if you like by opening this item in Content Editor: /sitecore/system/Marketing Control Panel/Taxonomies/Campaign group

This item comes from the item.master.dat file out of the box.

Another helpful tool is this SPE report that Jan Bluemink created, it lists all over-written IAR file items from a .dat file.

Hopefully this is useful info for anyone else who needs to cleanup IAR files and check the cleanup has worked correctly.

Below are some other useful links I found when working on this:

https://doc.sitecore.com/xp/en/developers/103/sitecore-experience-manager/work-with-items-as-resources.html
https://uxbee.eu/insights/items-as-resources-by-sitecore-part-3
https://jeroen-de-groot.com/2022/01/05/remove-items-from-resource-file/
https://gist.github.com/michaellwest/13e5a49b34340b9ebebdb83ff2166077

Using ChatGPT to write SPE

Catchy title huh? it Rhymes and everything.
Anyway, my Twitter timeline has been non stop about ChatGPT this, and ChatGPT that for the past few weeks – and I’ve seen it doing some pretty cool stuff.

I also keep hearing how it’s going to make Software Developers redundant, so it got me thinking about something I do every few days as a Sitecore Developer:

“Can ChatGPT write some decent Sitecore PowerShell Extensions scripts?”

TLDR: Yes it can. Not terrible anyway. Read on to find out more.

Wait, what is ChatGPT?

Chances are that you already know what it is else you probably wouldn’t be reading this, but just in case – ChatGPT is an impressive AI Chatbot created by OpenAI (backed by Microsoft & others) that can take inputs and provide some very comprehensive and usually pretty accurate answers.

The Tests – How did ChatGPT do?

So I fired up my browser, created an account on https://chat.openai.com/ and after waiting for it to become less busy for a while, I got started with asking for some SPE scripts:

ChatGPT Test 1 – Create me an Sitecore item

I thought I’d begin with something fairly simple – creating an item, it seemed like a nice place to start. It’s often something I need to script (perhaps usually to create a bunch of items). This is what came back:

My Score = 6/10

Other than the fact it seems to have tried to import SIF at the top of the script!?, Initially I was quite impressed as it looked pretty close to what I would write myself.
However when I tried to run the script in the SPE ISE (using the Sample Item Template Id) it was clear that it had some of the parameters incorrect. It had miss-understood how to use the -ItemType and had also tried to pass -Template in as a parameter and there is no such thing so it wasn’t quite right but not a bad effort. After correcting these issues this is the final working script:

ChatGPT Test 2 – Unlock all Items for a User

This time I went for something a little more tricky, unlocking items is something I’ve quite often need to do in the past – where users have gone on Holiday and left an lot of items locked for example. Here is what it came up with:

My Score = 9/10

Again it tried to import SIF at the top of the script, but other than that it actually created some working code. Amazingly similar to Marek’s SSE answer:

I was able to just remove the SIF imports at the top and run it and it seemed to work. It took a while (as I have a lot of test content locally) and there is no logging to show what happened but it didn’t error and did complete. Here is the working script:

ChatGPT Test 3 – Remove all old versions of an Item

This time I tried something I’ve had to do recently and took quite some time to write as was fairly complex. I wanted a script to remove all but the 10 most recent versions of an item but to always keep item versions that are 3 months old or less as well. Here is the script it came up with:

My Score = 3/10

These more complex scenarios and perhaps where ChatGPT starts to fall down a bit. It has tried to build two list (one of old versions and one of recent versions) and then remove the versions from the item and then add back the recent and old versions in memory.
This won’t work as it’s not updating the item itself and it’s not possible to set the Versions property on the item like this (instead $itemVersion| Remove-ItemVersion should be used). The logic is also not quite right here for sorting and filtering the versions. If you compare my script linked above there is quite a lot missing and wrong here due to the nuances with Sitecore item versions.

ChatGPT test 4 – Create me a package with a UI

A final test is also a task I’ve implemented recently. The scenario was a bit more complex than this but I thought I’d see how it got on with something similar:

My Score = 2/10

The script created here isn’t great to be honest. It didn’t quite get the properties correct for the Treelist UI. It has created the Package ok but the Add-ItemToPackage method does not exist, it should instead be using New-ItemSource.
Similarly it seems to have made up a function called Save-Package which should instead be Export-Package. Lastly it’s tried to offer me an link to the package to download it, which won’t work as it needs to either be an link to the default packages folder or use the Download-File option instead to present it to the user to download.
Interestingly when I tried to re-word my instructions to see if I could get it to use the Download-File method it actually tried to add items in a different (but still incorrect) way and tried to use the inbuild .NET Framework methods to download the zip to my browser which won’t work in the context of SPE ISE:

Conclusion

So am I worried I’ll be surplus to requirements anytime soon? In short – not currently no. The average of my (not so scientific test) was 5/10.

I think the nuances of a complex platform like Sitecore are hard for an AI powered tool to fully understand to an extent where it can automate code to an good-enough standard currently.

However, I think it could be a great starting point for an SPE script, especially for stubbing out the basics of something. It could also be really useful if Google or SSE fails you and you just can’t find any example code for a task you are working on. It is also very impressive that it can interpret what I’m asking for so well and write some half decent code.

I am not exactly sure how it works, but I’m guessing it’s injesting information from Sitecore Blogs, SSE, Sitecore Documentation Sites and reading the Dlls/Code online, as well as taking learning input from trainers and end-users.

If so then over time it’s going to become better and better and writing code and eventually I might be down the Job Centre wondering what happened.
If it manages to absorb Mark Kassidy’s brain then we will all be in Trouble…

Creating Preview / Draft Sites in SXA using SPE

A few weeks ago I was tasked with providing the ability for content editors to be able to have a preview (or draft) site for SXA sites when using the CM Url. The requirement was to be able to view ‘Draft’ content/versions of pages (without publishing them) for stakeholders to be able to preview them before they are published to the CD instance.

In an Traditional Sitecore setup this is pretty easy, you would just edit the site definition config in the Sitecore config with a patch file to change the database used to Master when on the CM.
For SXA this is a bit trickier as the Site definitions are dynamically built by SXAs site manager when Sitecore starts up (or the SXA Site Manager is ran again in Sitecore).

However after reading around it seems it was possible to achieve what I needed to do. I had to duplicate the existing Site Grouping definition item and change the database that it uses to Master. See an example below:

I also changed the environment to ‘Draft’ and added the following Patch file on the CM only, this tells the CM that the current environment is ‘Draft’ and to use Site Definition items with that environment setting:

 <sitecore>
    <settings>
       <!-- Environment name used by cm-->
       <setting name="XA.Foundation.Multisite.Environment" value="Draft" />
    </settings>
 </sitecore>

The last thing I did is change the name of the Site Grouping definition item and the title to include the word ‘Draft’. This was to make it clearer to content editors and stakeholders that they are editing/viewing ‘Draft’ content.

This results in each SXA site having two site Site Grouping definition items; the original (pointed at the Web DB) which the CD instance will use and the new Preview/Draft site that the CM instance uses:

Automating Creating of Draft Sites with SPE

As I needed to create Draft sites for quite a few sites I wrote an SPE script to do this for me:

Deleting Draft Sites

Should you need to remove your draft Sites (e.g during local testing) then I wrote a script for that too:

Site Ordering

One thing I discovered is that the order of the Sites in the SXA site manager matters. SXA will return the first one it matches based on Environment and the order defined here:
/sitecore/system/Settings/Foundation/Experience Accelerator/Multisite/Management/Sites

You therefore need to ensure the Draft site is above the original site grouping in the list. You can re-order these in the SXA Site manager using the up and down arrows but I had some issues with this not taking affect so instead I moved the draft sites up the list in the field they are stored in instead. As I had a lot of sites to do this for I actually ended up writing an SPE script to move all the draft sites to the top of the list:

Right-Click ‘Create Draft Site’

The above scripts were really useful to get the draft sites setup for the existing sites but I also needed a way to quickly add draft sites for new SXA sites in future. I decided an nice way or doing that would be using an SPE context menu script.

Here is the script, make sure you place it here: e.g: /sitecore/system/Modules/PowerShell/Script Library/{YOUR SCRIPT LIBRARY}/Content Editor/Context Menu:

Get CM Url Function:

Move Draft Site Function:

Experience Editor Notification

Again in order to make it clearer to content editors and stakeholders that they are editing/viewing ‘Draft’ content I decided an Experience Editor Notification would be useful. So again I turned to SPE to do this. I’m not sure if that many people know you can do this with SPE but it’s a really simple way to it with no C# code or config required:

Note that if you are using SPE 5 on Sitecore 9.02 or 9.01 then you will likely find there is a bug in the SPE config and you need to patch it like so to move the getPageEditorNotifications to the correct section of the config:

<configuration xmlns:patch="http://www.sitecore.net/xmlconfig/">
 <sitecore>
 <pipelines>
 <group name="ExperienceEditor" groupName="ExperienceEditor">
 <pipelines>
 <!--patch to add SPE 5.0 PageEditorNotifications in correct config group-->
 <getPageEditorNotifications>
 <processor type="Cognifide.PowerShell.VersionSpecific.Integrations.Pipelines.PageEditorNotificationScript, Cognifide.PowerShell.VersionSpecific"/>
 </getPageEditorNotifications>
 </pipelines>
 </group>
 </pipelines>
 </sitecore>
</configuration>

The end result of this work is that I now had Draft site functionality setup for all required sites, quick ways to add new draft sites as needed and it is clear to content editors that the draft site content is in use on the CM.

Dedicated Preview Instance considerations

Obviously for some scenarios you might have wanted to have a dedicated Preview/Draft instance with an specific url such as www.preview-mysite.com and publish to an ‘Preview’ database publication target. The config for the environment and hostname would then be used here instead. However for our purposes using the CM url is fine as the access is restricted and cuts down on costs of an additional instance in Azure.

Useful Links

As usual there were some really useful blog posts which helped point me in the right direction with this and figure out what I needed to do:

https://ggullentops.blogspot.com/2019/04/publishing-target-in-sitecore-sxa.html

https://getfishtank.ca/blog/setting-up-multiple-hosts-sitecore-sxa-site

https://blog.senktas.net/2019/10/20/sitecore-powershell-extensions-integration-with-experience-editor/

Hopefully others will find this information and the scripts I’ve written for setting up Preview sites in SXA useful in future.

p.s – if you wondering why the Harry Potter image; well apparently it took J. K. Rowling 13 attempts to finally get someone to publish the Philosopher’s Stone so it nearly remained in Draft and never published!

Removing Unneeded Sitecore Versions with SPE

It’s been a while since I had to look how best to manage Sitecore versions, but after seeing some performance issues related to lots of versions for some items on the CM this week It was time to revisit removing unneeded Sitecore versions.

Research

Amusingly my search turned up an SSE post with an answer from myself from about 5 years ago: https://sitecore.stackexchange.com/questions/507/limiting-version-numbers-copying-old-versions-to-archive-for-easy-access/525

Further Googling shows not much seems to have changed in that time. Some modules are however no longer supported or are compatible with Sitecore 9. I did find some links that gave me a few ideas though.

Our requirements were to find any items with more than 10 versions and then remove any versions other than the most recent ten unless an version is the currently published version or the version has been updated on the past 3 months. The idea behind this is that we wanted to remove only versions that were old and not updated recently but leave versions that are recent even if there are a lot of them for actively edited pieces of content.

The Script

Given the various modules available for managing versions are out of date and we only needed to this on an adhoc basis I decided to turn to trusty SPE to write a report which initially reported on items with 10 or more versions and then listed how many items to keep and how many to remove.

I was aware that Out of the box SPE has an report for item versions, so I used this as a basis for mine but added the check for if the version is the published version or has been edited within the past 3 months.

I then expanded on this to generate detailed reports of all the versions to keep and versions to remove.

Lastly added some options within the report to allow the item versions to be removed after the report has run an also if to archive the items to disk and/or the archive database table.

The result of this is that I could just run this in report mode first for analysis and then with the ‘Archive Versions & Generate Reports’ option selected to also remove the un-needed items.

Here is the script:

If you wish to change the number of months to a longer or shorter period you can update this variable value: $months = -3

When running the report with ‘Archive Versions & Generate Reports’ option selected it will ask you to confirm before removing items to be on the safe side:

Note that in newer versions of SPE the Remove-ItemVersion command archives items by default. However in the version we were using (5.1) this doesn’t happen so I added code to archive to the database before removing the item version.

Results

After the report has run an summary of results is shown and they can be downloaded in excel or csv format etc.

2 Other Spreadsheets are also automatically created and downloaded:

1. Versions to Keep – all the versions that will be kept for each item (these should correspond with the report above)

2. Versions to Remove – all the versions that should be removed for each item (these should correspond with the report above)

Running this in our production environment (after extensive testing) resulted in over 6,000 un-needed item versions being removed and over 7,000 being kept. This took around 40 minutes to run in total and the logs show info on what is happening.

Nearly halving our item versions means we should see some performance benefits and also it tidies up our database.

This also provides the added benefit of 3 reports we can reference if needed and the ability to restore versions from the archive if we need to in future for whatever reason. We plan to empty the archive in 6 months or so when we are sure these versions are not needed.

Hopefully someone else will find these scripts useful for managing item versions in Sitecore too. Feel free to edit it for your own use-case, please test carefully in a non-production environment though before running it.

Useful Links

There were some useful links I found regarding managing versions, including an SitecoreVersionPruner SPE script that I found after I’d written mine which looks really good:

https://github.com/danielrolfe/SitecoreVersionPruner

https://thesitecorist.net/2019/05/11/removing-old-versions-using-sitecore-scheduled-tasks/

https://briancaos.wordpress.com/2019/03/01/sitecore-delete-old-item-versions/

https://www.logicalfeed.com/posts/1195/sitecore-powershell-script-to-remove-all-language-versions-except-one

https://robearlam.com/blog/extending-sitecore-revolver-to-remove-old-versions

Removing Unneeded Tags from RTE fields with Sitecore PowerShell Extensions

This week I had an issue where some items imported from another system into Sitecore had invalid HTML in some of their RTE fields. These tags were breaking the SXA page content component that was being used.

I needed to a quick way to remove these tags without having to re-import the content. As usual SPE came to the rescue again and I was able to correct 100s of items in seconds.

The script has a function which takes an item and field name as a parameter:

Strip-InvalidTags -item $item -fieldName "Details"

I’m getting a number of profile items and calling this function in a loop. The function uses HTMLAgilityPack to look for ‘Script’ tag nodes (but can easily be modified to pick up different tags or multiple tags). If the script finds the tags is strips them and then saves the modified html back to the field.

Here is the script that I created:

Hopefully it’s useful for others who might need to do this.

Thanks again to the community for some useful ideas: https://sitecore.stackexchange.com/questions/20795/using-powershell-extensions-to-remove-empty-p-tags-from-all-rich-text-fields/20802

Enable and Disable Publishing Targets with SPE

It’s been a while since my last post as back February & March I was preparing for presenting at the Swiss Sitecore user group and I’ve been pretty busy since then, So I thought it’s time I shared some things that other might find useful from the past few months.

The client I’m currently working with needs to Toggle On and Off publishing Targets quite often in their UAT environment during performance testing. I posted a question a while ago on SPE about how I could do this but the answers about setting permissions for specific users were not really feasible for me.

So I wrote an SPE script to delete and restore the Publishing Targets, however our version of SPE (5.1) was too old for it to work (as I’m using the Restore-ArchiveItem function). We recently upgraded to SPE 6.2 and I was able to test it and it works great.

The Function

We have a Web and Pub publishing Targets so you will need to update these and the ids in the function below:

How to use It

Import the above function and then pass either ‘on’ or ‘off’ to it.

Import-Function Toggle-Publishing-Targets

Toggle-Publishing-Targets -state “on”

Thats it. You’ll get some feedback in the console to inform you when it’s removing them or restoring them.

It takes about 20 seconds or so to run in our UAT environment for some reason but it’s a nice quick way of doing this when needed. Hopefully this is useful for others too.

Scripting Sitecore Packages with SPE

This week I needed to automate the creation of a package to include a number of items that we want to backup before syncing content down from other environments. I needed to conditionally only backup certain test pages and components based on name.

I figured I’d be able to do this with Sitecore PowerShell Extensions and it turns out you can.

Here is a script based on the one included in SPE (/sitecore/system/Modules/PowerShell/Script Library/SPE/Core/Platform/Development/PowerShell Extensions Maintenance/Prepare Console Distribution).

The script creates a package with two sources:

Test Pages – pages within a specific folder
Test Components – matches components in sub-folders that have a specific name (sub-folders that start with ‘Test’)

It then provides the package for you to download like so:

To use this script update the meta data section at the top and the paths to include your own pages and components. Check the –InstallMode param is correct for what you want to do and also the filters for component name matches what you need, update it if not. You could include additional sources too by duplicating the $source = ‘x’ and $package.Sources.Add($source); lines.

Hopefully this is useful for others that need to do this in future.