User Profile Service and Synchronization services are showing running but profile synchronization not getting started: SharePoint 2010 & 2013

Yesterday our user were complaining that current changes in AD not getting replicated in SharePoint after initial inspection I found user profile not getting sync from last 2 days. I also tried to restart UPS and User profile synchronization service but no luck. I so tried to clean cache and restart timer services but it not helped.

When I checked log I am getting  below log event (Event ID 8301)

An exception occurred while updating addresses for connected app {7e90e5d3-268a-44c3-8020-d28b022b7d09_775ca572-a48f-4af9-9257-791e0129aa82}. The uri endpoint information may be stale. System.InvalidOperationException: The requested application could not be found at Microsoft.SharePoint.SPTopologyWebServiceApplicationProxy.ProcessCommonExceptions(Uri endpointAddress, String operationName, Exception ex, SPServiceLoadBalancerContext context)at Microsoft.SharePoint.SPTopologyWebServiceApplicationProxy.ExecuteOnChannel(StringoperationName, CodeBlock codeBlock) at Microsoft.SharePoint.SPTopologyWebServiceApplicationProxy.GetEndPoints(GuidserviceId) at Microsoft.SharePoint.SPConnectedServiceApplicationAddressesRefreshJob.Execute(Guid targetInstanceId)

Log files are showing below entry  with High Level in User profile category

User Profile Application: SynchronizeMIIS encounters an exception: System.NullReferenceException: Object reference not set to an instance of an object. at Microsoft.Office.Server.UserProfiles.UserProfileImportJob.<>c__DisplayClass2.<IsTimerJobRunning>b__1() at Microsoft.SharePoint.SPSecurity.<>c__DisplayClass5.<RunWithElevatedPrivileges>b__3()

at Microsoft.SharePoint.Utilities.SecurityContext.RunAsProcess(CodeToRunElevated secureCode)

at Microsoft.SharePoint.SPSecurity.RunWithElevatedPrivileges(WaitCallback secureCode, Object param)

at Microsoft.SharePoint.SPSecurity.RunWithElevatedPrivileges(CodeToRunElevated secureCode)

at Microsoft.Office.Server.UserProfiles.UserProfileImportJob.IsTimerJobRunning(UserProfileApplicationJob timerJob)

at Microsoft.Office.Server.Administration.UserProfileApplication.SynchronizeMIIS()

at Microsoft.Office.Server.Administration.ILMProfileSynchronizationJob.Execute()

 

It means we are getting null reference exception when FIM management trying to connect to SharePoint. Most of time this is related to timer job that will be created when we set UPSA to Active Directory import . But in our case user profile sync was running fine and when I read about this it show this property can put UPSA in unstable condition so if some how it toggle it could be create issue. I checked using the below command but it showed it is already set to false

$upsa = Get-SPServiceApplication -name "User Profile Service Application"
$upsa.NoILMUsed
But since we have custom code in our farm so thought it to reset by appending below command $upsa.NoILMUsed = $true
$upsa.Update()
$upsa.NoILMUsed = $false
$upsa.Update()


After that i hit user profile sync and it work well.

References

https://support.microsoft.com/en-us/kb/2624198

http://blogs.msdn.com/b/ronalg/archive/2015/02/17/sharepoint-2013-external-fim-issue-missing-manager-property.aspx

SharePoint Workflow 2013 not working throwing all 0 co-relation ID error page

In recent past our SharePoint 2013 users were complaint that SharePoint workflows are not working and when ever they click on workflow column (this column created automatically by SharePoint workflow) is navigating to error page that displaying all 0 guid. 

We looked in ULS logs but did not find any specify information. In SharePoint 2013 workflow architecture is very different. SharePoint 2013 workflow run on service bus and it have its own separate databases and architecture.

Open the Power shell management in Workflow server where have workflow manager installed. run get-WFFarm command to get certificate and endpoints related information. Make sure certificate are not expire. Run get-WFFarmStatus tshin will return the WorkFlow endpoint status. here you could get a warning related to Workflowservicefrontend and this could be show stopped but important is workflowservicebackend that should be running if not than we can run that suing shell command.

here it seems that every thing is fine.

What could we next? than event viewer can help us go to event viewer and expand Application and service log here you will find a Microsoft-Workflow, Expand this and select operational here you will get several warring like below , so this is the issue.form the details it showing that account not have permission to execute the Stored procedure in workflow database.

Now we understand what is the issue. to fix this issue we need to log in SharePoint SQL serve with the account that have sysadmin permission. so after log in the server and fire the SQL management studio, Expand databases here we find the same name database if we expand workflow database than we can find this stored procedure as well.

Log Name:      Microsoft-Workflow/Operational

Source:        Microsoft-Workflow

Date:          11/18/2014 5:39:54 AM

Event ID:      18

Task Category: (65516)

Level:         Warning

Keywords:      Dispatcher,Management

User:          Domain\Service account

Computer:      Machinename.Domain.net

Description:

Failed SQL command after 1 tries with error '229'. Exception: System.Data.SqlClient.SqlException (0x80131904): The EXECUTE permission was denied on the object 'InsertTrackingAndStatus', database 'OPSPP_WorkflowManager_InstanceManagementDB', schema 'dbo'.

   at System.Data.SqlClient.SqlConnection.OnError(SqlException exception, Boolean breakConnection, Action`1 wrapCloseInAction)

   at System.Data.SqlClient.TdsParser.ThrowExceptionAndWarning(TdsParserStateObject stateObj, Boolean callerHasConnectionLock, Boolean asyncClose)

   at System.Data.SqlClient.TdsParser.TryRun(RunBehavior runBehavior, SqlCommand cmdHandler, SqlDataReader dataStream, BulkCopySimpleResultSet bulkCopyHandler, TdsParserStateObject stateObj, Boolean& dataReady)

   at System.Data.SqlClient.SqlCommand.FinishExecuteReader(SqlDataReader ds, RunBehavior runBehavior, String resetOptionsString)

   at System.Data.SqlClient.SqlCommand.CompleteAsyncExecuteReader()

   at System.Data.SqlClient.SqlCommand.EndExecuteNonQueryInternal(IAsyncResult asyncResult)

   at System.Data.SqlClient.SqlCommand.EndExecuteNonQuery(IAsyncResult asyncResult)

   at Microsoft.Workflow.Management.ExecuteNonQueryAsyncResult.OnEndExecuteSql(IAsyncResult result)

   at Microsoft.Workflow.Management.ExecuteSqlAsyncResult.EndAsyncResult(IAsyncResult result)

   at Microsoft.Workflow.Common.BackoffRetryAsyncResult.IsolateWithRetry(IAsyncResult result)

   at Microsoft.Workflow.Common.AsyncResult.AsyncCompletionWrapperCallback(IAsyncResult result)

ClientConnectionId:50d4777e-0f3e-4921-a1bc-a383cc417348  Command Details:   SQL Text          : InsertTrackingAndStatus

  SQL Parameters    :

    > @StatusRecords = Microsoft.Workflow.Management.SqlDataAccessProvider+InstanceStatusValuesCollection

    > @MetadataWithInstanceId = Microsoft.Workflow.Management.SqlDataAccessProvider+MetadataValuesWithInstanceIdCollection

    > @Variables = Microsoft.Workflow.Management.SqlDataAccessProvider+VariablesCollection

    > @TrackingRecords =

    > @StatusHistoryRecords =

    > @DebugTraceRecords = Microsoft.Workflow.Management.SqlDataAccessProvider+DebugTraceRecordValuesCollection

    > @RollingStatusHistoryWindowSize = 1000

    > @RollingDebugTraceWindowSize = 5000

Event Xml:

<Event xmlns="http://schemas.microsoft.com/win/2004/08/events/event">

  <System>

    <Provider Name="Microsoft-Workflow" Guid="{B2885F6E-231C-43FF-BBEA-7516148FF6FE}" />

    <EventID>18</EventID>

    <Version>0</Version>

    <Level>3</Level>

    <Task>65516</Task>

    <Opcode>0</Opcode>

    <Keywords>0x4000000000000210</Keywords>

    <TimeCreated SystemTime="2014-11-18T10:39:54.821991300Z" />

    <EventRecordID>1643486011</EventRecordID>

    <Correlation ActivityID="{C63E7DEC-AB83-43C3-9721-AEBB3F0BD3C9}" />

    <Execution ProcessID="6364" ThreadID="5684" />

    <Channel>Microsoft-Workflow/Operational</Channel>

    <Computer>SBYVMNTSHP03P.Domain.net</Computer>

    <Security UserID="S-1-5-21-369997941-647960827-447208795-648028" />

  </System>

  <EventData>

    <Data Name="tryCount">1</Data>

    <Data Name="errorCode">229</Data>

    <Data Name="command">  SQL Text          : InsertTrackingAndStatus

  SQL Parameters    :

    &gt; @StatusRecords = Microsoft.Workflow.Management.SqlDataAccessProvider+InstanceStatusValuesCollection

    &gt; @MetadataWithInstanceId = Microsoft.Workflow.Management.SqlDataAccessProvider+MetadataValuesWithInstanceIdCollection

    &gt; @Variables = Microsoft.Workflow.Management.SqlDataAccessProvider+VariablesCollection

    &gt; @TrackingRecords =

    &gt; @StatusHistoryRecords =

    &gt; @DebugTraceRecords = Microsoft.Workflow.Management.SqlDataAccessProvider+DebugTraceRecordValuesCollection

    &gt; @RollingStatusHistoryWindowSize = 1000

    &gt; @RollingDebugTraceWindowSize = 5000

</Data>

    <Data Name="exception">System.Data.SqlClient.SqlException (0x80131904): The EXECUTE permission was denied on the object 'InsertTrackingAndStatus', database 'OPSPP_WorkflowManager_InstanceManagementDB', schema 'dbo'.

   at System.Data.SqlClient.SqlConnection.OnError(SqlException exception, Boolean breakConnection, Action`1 wrapCloseInAction)

   at System.Data.SqlClient.TdsParser.ThrowExceptionAndWarning(TdsParserStateObject stateObj, Boolean callerHasConnectionLock, Boolean asyncClose)

   at System.Data.SqlClient.TdsParser.TryRun(RunBehavior runBehavior, SqlCommand cmdHandler, SqlDataReader dataStream, BulkCopySimpleResultSet bulkCopyHandler, TdsParserStateObject stateObj, Boolean&amp; dataReady)

   at System.Data.SqlClient.SqlCommand.FinishExecuteReader(SqlDataReader ds, RunBehavior runBehavior, String resetOptionsString)

   at System.Data.SqlClient.SqlCommand.CompleteAsyncExecuteReader()

   at System.Data.SqlClient.SqlCommand.EndExecuteNonQueryInternal(IAsyncResult asyncResult)

   at System.Data.SqlClient.SqlCommand.EndExecuteNonQuery(IAsyncResult asyncResult)

   at Microsoft.Workflow.Management.ExecuteNonQueryAsyncResult.OnEndExecuteSql(IAsyncResult result)

   at Microsoft.Workflow.Management.ExecuteSqlAsyncResult.EndAsyncResult(IAsyncResult result)

   at Microsoft.Workflow.Common.BackoffRetryAsyncResult.IsolateWithRetry(IAsyncResult result)

   at Microsoft.Workflow.Common.AsyncResult.AsyncCompletionWrapperCallback(IAsyncResult result)

ClientConnectionId:50d4777e-0f3e-4921-a1bc-a383cc417348</Data>

  </EventData>

</Event>

Now we need to check whether the account under the Workflow services are running have sufficient permission. right click the database and select properties.

Here we can see all the account and roles that have permission on this database.

but here is gotcha we don't know what is the right permission that need to give although we can give dbowner permission to workflow account but that could give more than required permission.

As per MS article and understanding this is not recommended to give direct permission to any account so we have another way here you will find  role. this  is the key to solve the issue. Open the Query window and check selected database is “XXXXX-InstanceManagementDB” and write GRANT EXECUTE TO WFServiceoperators hit execute.

Issue is fix ? partially yes :) if we check event log warning is disappear form the event log and if we again go our site collection and try to reproduce the issue it will change with another database  that is XXXX-ResourceManagementDB” so we need to follow the same way and execute the command for the DB as well and grant permission to role again.

now close browse fire a new instance and try to reproduce the issue. this is the way i solve the issue hope it will help you as well.

Resource Throttling

This topic is quite interesting in many respect. In recent past few days i was learning about this feature and what this feature offer is quite interesting.

I read many article and they explain this quite good but i just to include all my experience this feature offer.

Throttling is a administrative feature that provided in SharePoint 2010 and continue in SP2013.

Why we need this

In many business case scenario when we have a big list  more that 10,000 + item (Managing big list is separate  topic we discuss about this some other time) user request that they want to get more than 5000 item but due to resource constraints\Performance reason SharePoint take intelligent OOB decision and not get more and display a error message. this is very generic message.

The list view threshold does not apply simply to the number of results returned by our query. Instead, it restricts the numbers of database rows that can be accessed in order to complete execution of the query at the row level in the content database.

So obvious solution comes in our mind is enabling throttling(enabling for big query) for particular time period. Before you make any changes  Here we need to first identify the time where our server have least load this is critical decision if your SharePoint is getting access by the all the continent and user base is good enough like  80-90,000.

But  here story is not complete this setting is also going to effect the “List View Lookup Threshold” so in short if we create a view and we are using the lookup columns than we can get result in view up to  specified numbers by default it is 8 (but it count create by and modified by also a lookup columns so actually this is 6).

List view throttling is showing results on view that have internal lookup columns. if we say 8 is the threshold value it no mean that we can use 8 lookup columns it means that query can fetch the result using specified joins. So if you have multiple tables with lookup columns and want to show result in one view please plan.

we can increase the number but take this decision with good thought because first it going to consuming lots of resource and second that would annoying is if We increase limit for one business case than you can get next query to again increase it for another business case :) so be prepare. So rather that increasing this i will leave it default.

One more thing that is i found associated with this setting is Export to Excel option in list ribbon. Suppose that you have a list view that have multiple internal lookup columns and thousand items now when use can see the items in particular view but when he/she not able to export the list item in excel using the option form SharePoint ribbon.(make sure you have contributor right , if you are administrator this is going to work for you).

 Where we can enable this

We can change the setting from the Central admin. GO to manage web application and select particular web app.from the ribbon select General setting and choose resource throttling.

 

There is also some interesting idea to manage throttling using Indexing.

http://en.community.dell.com/techcenter/sharepoint-for-all/b/blog/archive/2014/04/01/resolving-the-list-view-threshold-when-migrating-to-o365

2013 Workflow option not showing in SharePoint Designer

 

In last day user reported that in SharePoint designer 2013, workflow option for 2013 not showing in dropdown box.

We find that SharePoint 2013 workflow options available for other web application's site collections.

Although there could be many reason because in SharePoint 2013 workflow have different architecture but since workflow are available for the other web application so we were sure that  workflow is fine something is wrong with the site collection or association, so our first place to check service association with webapplication

follow below steps

a. In central admin go to manage web application in Application management heading.

b. click on the web application where you are getting issue

c. in ribbon click in   button it will open a window where we can see all the service application proxy associated with troubled web application.

d. scrawl   down and you check whether below option checked , in our case workflow service application proxy was not checked.

e. if in your case it is uncheck that go to top of the same window and from the drop down select custom option it will show you all the option so you need to check all required service proxy that your web application want to consume. this done

f. close your already open designer and aging open check Workflow 2013 option will be there. Hope this help you.

Redirect SharePoint http request to Https

In our On Premise SharePoint environments we are using https protocol. Because this protocol use certificate so this give us more security but generally user don’t care  about protocol. for example user always type xyx.abc.com and hit enter now this is up to the browser because there is port enclosed so browser took default http port 80  but because environment set up on https so port would be 443. :( it not going to work.

So how can we resolve this issue? Suppose we have a xyz.abc.com and our designer put lots of efforts to make it as interne site so user can get all information about organization. But remember our all environment based on SSL so i we don't want taht every time use put full address like https://xyz.abc.com so we decide to redirect out all request that coming using http will be redirect to https.

here we complete out text now hands on :)

This is going to  be handle by IIS so we need to work there first.

1. Open IIS manager and create a dummy Application pool for example request redirect App, so case if there is another site need to redirect i can use this.

Right click on application pool and window will prompt fill the desired Name.

2. After this we need to create a dummy site that take out http request and redirect to https

So right click on sites node and select Add site

Now first fill site name . choose name that you can understand vey clearly.

Now click on Select just beside on Default app pool and select our app pool that we created in previous step.

Now we need to select a physical path this path should be in inetpub but out side of wss ,becuase if we create a folder in wss SharePoint will throw permission level error so create out site a folder where IIS will keep it xml files.

After that, don't change type {http} and port 80 because if you change port user have to send request with port and we want to remove his pain :)

now in host name just put the name with out protocol “xyz.abc.com”

Click ok . congratulation you complete good work but still you need to put some changes in this newly created site’s settings.

If now you will try to access this site you will get a forefront protection error. and we also not make entry to redirect on our desired site.

3. select our newly created site  from let pane and you will fine these option in middle window

Now we have to take care about two settings only

first get rid of forefront protection error issue

and second redirect to out site

so let take first thing first

select      this option and remove

 

FSSP username filter this is fro Microsoft forefront protection for SharePoint.

after that double click http redirect option .

select check box “redirect request to this destination ”. In textbox fill full URL including the https. Leave below checkbox unselected .

Don't forget to apply from right pane.

All is set to work.

Now open your browser and access site by using http protocol.

Rename Host Based site collection

As administrator several time we require to move the site collection from one environment to another using the content DB backup and restore.URL those have relative path take by SharePoint itself but host based site collection require additional attention because those URL are not depend on web application.

We have 2 way either in new environment we took back of host based site first and restore with nee URL

or we can use few command  and change the name

$site = get-spsite "URL NAME"

$site

$site.rename("New URL")

OR

stsadm -o renamesite -oldUrl http://HostHeader_1 -newUrl http://HostHeader_2

Hope this help you.

Accidently delete TFS default group

Accidently we deleted default contributor group.

Effect : Developer who have contributors permission not able to see the solution in project explorer

Solution: Below step is one of the way through be fix the issue.

1. Connect to TFS using the TFS administrator credentials

1. Click on Settings it will navigate to Team project and Team project Collection

1. Under Team Project click on Group Membership and it will prompt a window to create a Group , fill the value like Contributor and description and click OK
2. Now click on Security and click on ADD this will help you add newly create group in the Team Project along with the desire permission. Make sure we select Team Foundation Server Group radio button.

From the permission group select the permission and click close

1. Now we need to give permission on Work Item and iteration

Select the top and click on security and Select the group and from the permission add permission click close do same with Iteration.

Through this we can control if on different area/iteration different permission required. By default the inherit.

Now right click on Project name from the Source control solution from the advanced choose security Because we are not have complex TFS architect

Now from the properties of the project click on add and choose the group that you want to provide the permission now from the permission section choose the permission .